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1. Introduction 



This report describes the development of the Polish 
Proficiency Test (PPT) , a standardized, national ly-normed test of 
listening and reading comprehension for English-speaking learners 
of Polish. The PPT was developed by the Center for Applied 
Linguistics (CAL) under the auspices of the United States 
Department of Education (Grant No. PO17A10051) with the 
cooperation of numerous linguists and Polish language experts 
from a variety of academic institutions across the country. The 
project was initiated in October 1991 and completed in September 
1992. The test was developed in response to the lack of a 
standardized proficiency test in the Polish language teaching 
field and an increased need for such a test. 

This introductory chapter provides the background to the 
development of the PPT, beginning with a description of the 
relationship between Polish language teaching and the Polish 
Proficiency Test, and concluding with the description of the 
structure, format and content of the Polish Proficiency Test. 

1-1 Polish Language Teaching and the ppy 

Before the project was proposed, a survey was conducted by 
CAL staff of tests available in Polish. The results indicated 
that the Polish language teaching field was handicapped by the 
lack of a nationally-available proficiency test. Only three 
Polish exams were identified in the CAL survey. One of the 
exams, the Lektorek Razor Blade Reading Proficiency Battery, is a 
computer-based evaluation of reading and grammar. Owing to its 
mode of administration, this Polish test is not accessible for 
institutions that do not possess computerized testing 
capabilities. The second test was the Polish Proficiency 
Examination, which was found to be more of an achievement test 
based on the materials taught in the specific academic program 
which developed the test than a genuine proficiency test. The 
third was the Polish Second Language Graduation Proficiency Test. 




/ 



The intended use of this test was as an exit criterion of the 
foreign language requirement at the test developer's own 
institution. It was not intended for national use. Thus, none 
of the three tests is easily accessible and available for use 
nation-wide. 

The PPT now makes available to the Polish language teaching 
profession a standardized and truly proficiency-oriented test 
that is applicable to any Polish language program regardless of 
its curriculum. The PPT can be used to meet a variety of needs 
of Polish teaching programs, including the following: 

• Program Placement; i.e., institutions can use the test as 
a tool to determine the most appropriate instructional levels for 
incoming students with previous language training; 

• Program Evaluation; i.e., language programs can use the 
test to assess their instructional effectiveness by examining the 
relative standing of their own students compared to nation-wide 
test performance; 

• Exit Criterion; i.e., individual programs can establish 
their own required exit score to help them determine if a student 
is eligible to leave their program or to get credit for it; 

• Selection Criterion; i.e., schools, study abroad 
programs, business or government agencies can use the test to 
select the best applicants from a pool of candidates who have 
acquired their knowledge of the language in a variety of ways; 

• Credit by Exam; i.e., institutions can offer their 
students the option of fulfilling a language requirement for a 
certain program on basis of the PPT score; and, 

• Research; i.e., the PPT can be used as a common measure 
in research projects. 

1-2 Structur e of the PPT 

The PPT has three sections: Listening Comprehension, 
Reading Comprehension and Structure. All items on the PPT are 
four-option multiple choice. 

In the Listening Comprehension section, examinees hear a 



passage in Polish, followed by a question about it in English. 
Both the stimuli and the question are heard from a 
professionally-recorded Master Tape. After hearing the English 
question, examinees then choose one of the four answers printed 
in English in the PPT Test Booklet and mark their choice on the 
machine-readable PPT Answer Sheet. 

In the Reading Comprehension section, examinees are 
presented with a variety of Polish texts each followed by a 
question about it in English. The examinee is provided with four 
response choices. The Polish text, the English question and the 
four choices are all printed in the PPT Test Booklet. 

The Structure section h* s two parts: filling in the blanks 
(cloze) and error identification. In Part One, there are five 
Polish passages, each with five missing portions. The missing 
portions are indicated by an underlined item number. Beneath 
each passage are five items labeled with item numbers 
corresponding to those in the passage. For each item, examinees 
are required to choose one of the four options which completes 
the corresponding missing portion of the passage in a meaningful 
and grammatically correct way. In Part Two, examinees are 
presented with three written Polish passages, each with five 
numbered lines. In each numbered line, there are four underlined 
words, marked A, B, C, D. Examinees are required to identify the 
one underlined word which must be changed in order for the line 
to be grammatically correct. Examples of these items are 
included in the PPT Examinee Handbook . 

The design of PPT is intended to make it flexible. A 
program may choose whether to administer the Structure section 
depending on whether or not grammar is emphasized in instruction. 
Although normally administered in a single session, if required 
by class scheduling constraints, the PPT may be administered in 
two separate sessions. 

Table l provides an overview of the organization of the 
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PPT. 



Table 1 
Organization of the PPT 



Section 
Listening 



Total 
Time 
40 min 



Total Number 
of Items 
50 



Format of 

Stimulus 

Utterances 

Dialogues 

Monologues 



Number 
of Items 



10 
25 
15 



Reading 
Comprehension 



50 min 



50 



"Non-linear Text" 
"Passages" 



15 
35 



Structure 



30 min 



35 



Cloze 

Error Identification 



25 
10 



Although in general items on the PPT are arranged in order 
of increasing difficulty, they are also grouped according to the 
format of the item stimulus. For example, items in the Listening 
Comprehension section are grouped into three parts according to 
the format of the listening passage. Items in the first part, 
"Utterances," present the examinee with the speech of a single 
speaker excerpted from a longer dialogue or conversation. Items 
in the second part present short dialogues between two speakers. 
Items in the third part present examinees with speech found in 
naturally-occurring monologues, such as those from radio 
announcements or news broadcasts. 

Although there are no separately designated parts in the 
Reading Comprehension section, there are a variety of text types 
that are used. These may be classified into two main groups. 
The first is "Non-linear Text," which refers to reading stimulus 
that is non-prose in format; i.e., whose form falls outside the 
normal conventions of sentence, paragraph and text structure. 
Examples include the writing found in signs, book titles, on 
stamps, in schedules, on identification cards and the like. The 
second may be termed "Passages." These texts present examinees 
with prose and expository text following normal conventions of 
sentence and paragraph structure. The classification of the 
reading stimuli into these two categories is somewhat arbitrary, 



since many passages contain both (e.g., newspaper 
advertisements). In Table 1, an attempt has been made to 
classify each stimulus text by its primary characteristic. 

Items in the Structure section are of a cloze (f ill-in-the- 
blank) type and error identification type. Both types focus on 
aspects of grammar and structure. The cloze type, which is Part 
One, has five passages, each containing five test items. The 
error identification type, which is Part Two, has two passages, 
each containing five test items. 

1.3 Content of the PPT 

The development of the PPT followed strict guidelines in 
terms of the test content covered. All Polish materials on the 
PPT are drawn from authentic sources; i.e., Polish language 
materials prepared for Polish native speakers. Sources for 
listening passages included Polish movies, recordings of 
conversations, and radio and television programs. For passages 
for reading and structure, sources included Polish magazines, 
journals, newspapers, books, informational brochures, merchandise 
packaging, tickets, public signs and personal documents. 

Every effort was made to include as wide a range as possible 
of the various social and institutional interactions that would 
be most likely encountered in real-life language-use situations 
in a Polish-speaking environment. The content covered on the PPT 
can best be described in terms of topic areas (subject of the 
listening or reading passage) . Table 2 presents an overview of 
the topic areas (content) of the PPT covered in the listening and 
reading sections of the test. The first column lists the topic 
areas covered, the second lists the number of listening passages 
for which that topic could be described as the main topic of the 
passage, and the third lists the same for the reading passages. 





Table 2 

Topic Areas Covered on the PPT 


Topic Area 


Listening Section 


Reading Section j 


Art 


1 




Business 


1 


2 


Consumerism 


7 


6 


Dates 


1 


— 


Economics/ the Economy 


3 


5 


Education 


3 


6 


Entertainment 


5 


5 


Everyday Life 


3 


1 


Family 


3 


1 


Food 


2 


— 


Geography 


— 


1 


Government 


1 




Health 


5 


3 I 


History 


— 


1 


Housing 


— — 


1 


Information Listing 


— 


1 


Politics 


3 


1 


Social Exchange 


3 


— 


Social Life 




2 


Society 


1 
1 




Sports 


4 


2 I 


I Transportation 


3 


7 


1 Weather 


1 




| Other 
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Each passage in the listening comprehension section was also 
classified by the test developers in terms of the primary 
speaking function of the passage. In many cases, there are 
several speaking functions- Each item, however, was assigned vo 
only one primary function, the one most closely associated with 
getting the passage's item correct. Table 3 gives an indication 
of the distribution of the speaking functions found in the PPT 
listening comprehension section. 



Tabic 3 1 
Speaking Fraction in die 1 it-i^g Section 1 


Speaking Function 


Number of Items 


Complaining 


6 


Describing 


1 


Expressing an opinion 


2 


Giving advice 




Giving information 


19 


Making an inquiry 


11 


Interviewing 




Making a comment 




Making a request 




Making a purchase 




Making an introduction 




Making an invitation 




Negotiating 




H Persuading 




H Showing preferences 





1.4 Test Ad ministration Time 

The three sections of the PPT may be administered in either 
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one or two sessions, with the administration of the structure 
section being optional. In a two-day session, the listening 
comprehension section and the structure section (if included) are 
administered at the first session, with the reading section 
administered at the second. Exclusive of passing out test 
booklets, filling in personal background information on the 
machine-readable answer sheet, and taking care of other 
administrative matters, the listening comprehension section 
requires approximately 40 minutes, the reading comprehension 
section 50 minutes, and the structure section, 30 minutes. If 
given in one sitting, the total testing time is about two hours 
and a half. If administrative matters are taken care of before 
the testing session begins, the PPT can be administered in one 70 
minute and one 50 minute class period. Two 80 minute periods are 
adequate for handling the complete administration of the PPT. 

1.5 PPT Test Materials 

The operational program of the PPT consists of the following 
materials (copies of these materials can be found in the 
appendices as indicated) : 

PPT Test Booklet 

The PPT Test Booklet contains all test instructions, the 
options for the Listening Comprehension items, the Polish text, 
English questions and options for the Reading Comprehension 
items, and the Polish text and options for the Structure items. 
Each examinee receives a copy of the test booklet, which is 
collected by the test supervisor at the end of the test. Because 
the test is secure, only the cover of the test booklet is 
reproduced in Appendix A: PPT Test Booklet Cover. 

PPT Master Tap p 

The PPT Master Tape is a professionally recorded audio tape 
containing the listening stimuli and the English questions for 
the Listening Comprehension section of _he PPT. Normally, two 
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copies of this tape are sent to institutions ordering the PPT, 
one to play and one as a back-up. 

PPT Answer Sheet 

This is a machine-readable answer sheet, specially designed 
for the PPT program- In addition to spaces for examinee 
responses to items, it contains areas to collect examinee 
background data- A copy of the answer sheet can be found in 
Appendix B: PPT Answer Sheet. 

Test Administration Instructions ( S upervisor's Manual! 

This is a short manual for test supervisors containing all 
the necessary instructions for administering the test. In order 
to ensure that the PPT is administered under standardized 
conditions, all instructions to examinees are to be read verbatim 
from this manual. A copy of this manual is contained in Appendix 
C: PPT Supervisor's Manual. 

PPT Examinee Handbook 

This handbook is intended to familiarize examinees with the 
test before they take it. It reviews basic information about the 
test and gives sample test items for the test. Each examinee 
receives a copy of the Examinee Handbook from the test supervisor 
prior to taking the test. A copy of the preliminary handbook is 
contained in Appendix D: PPT Examinee Handbook . Before the 
final copies are printed, CAL is waiting on some data from 
several government agencies that will enable preliminary 
conversion tables between the PPT and the ILR Proficiency Scale. 
Otherwise, the Examinee Handbook is complete. 

Test Interpretation Manual 

A test interpretation manual has been prepared for the PPT. 
The Test Int erpretation Manual helps test users interpret test 
scores so that they can be meaningfully used in their programs. 
In addition to a brief description of the PPT, the Manual 
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contains technical information on the tests and test 
interpretation tables. A copy of the preliminary manual is 
contained in Appendix E: PPT Test Interpretation Manual . Again, 
CAL is waiting on some data from several government agencies that 
will enable preliminary conversion tables between the PPT and the 
ILR Proficiency Scale before printing the final version. 
Otherwise, the Test Interpretation Manual is complete. 

Miscellaneous Ope rational Test Forms 

There are several forms that are used in the operational 
testing program, sample copies of which may be found in Appendix 
F: Forms for the Operational Testing Program. 

Test Order Form 

This form is used by test users to order the test from 
CAL. 

Checklist 

This form is used to ensure that PPT test materials are 
returned to CAL. 

Supervisor's Report Form 

This form is used by test supervisors to record any 
problems that may have occurred during test 
administration. 
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2. Project Start-Up 



The initial phase of the PPT project involved a great many 
efforts to ensure a sound theoretical and methodological 
framework for the development of the test items. This chapter 
describes the initial steps of the test development project. It 
encompasses both logistic issues and theoretical concerns. 

2.1 Test Development Committees 

The day-to-day activities of the project were coordinated 
and directed by a team of CAL staff members* Charles W. 
Stansfield served as the Director of the Project, with Dorry Mann 
Kenyon serving as Associate Project Director and Xixiang Jiang as 
Project Coordinator* 

To assist in the development project, two test development 
committees were formed. The first was the External Advisory 
Committee, consisting of Polish linguists and Polish language 
experts from across the nation. Many of these were professors in 
major university-level Polish programs. Below is a list of the 
members of this committee and their affiliations. 
External Advisory Committee 

Waldemar Martyniuk Stanford University 

Barbara Niemczyk Dickinson College 

Leonard A. Polakiewicz University of Minnesota 



Members of this committee were asked to review 
specifications for the test, review the test forms before field 
testing, and to review revisions made to the test form. As 
possible, these committee members also helped coordinate field 
testing of the PPT at their respective institutions. 

The second committee was the Local Development Committee. 
Members of this committee were Polish language specialists and 



Robert A. Rothstein 
Alexander Schenker 
Oscar E. Swan 



University of Massachusetts 
Yale University 
University of Pittsburgh 
Defence Language Institute 
State University of New York 



Krystyna Wachowicz 
Joanna Radwanska Williams 



experienced language instructors. Members of this committee and 
their affiliations are listed below. 



Local De velopment Committee 



Dobrochna E. Dyrcz-Freeman* 



(Formerly) Princeton University 

(Formerly) CAL 
CAL 

Foreign Service Institute 

CAL 



Rafal M. Kwiatkowski* 
Grazyna Z. Privorotsky 
Krystyna M. Sadlowska 
Waldemar E. Walczynski 



* primary test item writers 

Members of the local development committee met to draft the 
initial test specifications, developed the test items, reviewed 
items under development, reviewed the test form before field 
testing, and served as consultants when special problems arose. 

The two item writers were trained by CAL staff. They were 
in regular contact with CAL staff throughout the project. They 
were tasked to find suitable listening and reading passages from 
authentic sources and draft items to test listening and reading 
comprehension, as well as to draft the structure items. 

In addition to these two working committees, four Polish 
native speakers were involved in recording the PPT Master Tape. 
Given their various areas of expertise and experience in 
teaching, testing and with the Polish language, all members of 
the project committees contributed to its successful completion. 

2.2 Initial Committee Meeti ngg 

Each committee met within two months of the project start- 
up. The results of their initial meetings, which set the course 
of the project, are described below. 



The Local Development Committee met in mid October, 1991, to 
develop the test specifications, to set down the guidelines with 
which the test item writers would work, and to discuss issues 
relating to the development of the test items. Several test 



2.2.1 



Initial Local A dvisory Committee Meeting 



development concerns were addressed at the meeting regarding 



target levels, test domains, test format and the number of items 
in each section. Members of the Committee conferred for a day 
and concluded with a general agreement on the issues considered. 

In view of the difficulty of the language itself and the 
limited nature of the existing teaching materials, it was 
recognized that the target level of the test should be between 0+ 
and 2+ on the ILR scale, corresponding to Novice-Mid/High to 
Advanced-High on the ACTFL scale. In consideration of the fact 
that most Polish language teaching programs do not continue 
beyond the second year and that students in area studies at the 
graduate level usually enter beginning Polish programs at the 
undergraduate level, it was agreed that the envisioned target 
examinee group for this test include Polish language students in 
their first or second year of college and government or business 
professionals in intensive language training programs. 

It was also agreed that the PPT, as a proficiency test, 
would measure the examinees' ability to function in a real 
Polish-speaking environment. Therefore, it would not be designed 
to accommodate any specific Polish language teaching text or 
curriculum, nor would any sort of achievement test be appended to 
this proficiency test. Acknowledging that the knowledge of 
syntactic structures is an integral part of communicative ability 
in the Polish language, it was determined that this test would 
include a structure section. In addition, it was acknowledged 
that the materials used in the items should be authentic; i.e., 
the type of language materials that are produced by native 
speakers for use by native speakers in their native environment. 
The authenticity of the test was expected to promote and lead the 
trend in the language teaching field toward the use of real 
language to achieve communicative competence in the target 
language. 

In terms of the test item format, it was conceded at the 
meeting that a four-option multiple choice format be used 
throughout the test, in conformity with the current trend of 
large scale tests. It was also agreed that structure section be 



comprised of two parts: a cloze format and an error 
identification format. As for the length of the test, it was 
suggested that the whole test contain between 125 and 150 items, 
with roughly equal numbers in listening, reading and structure 
sections. 

Apart from the major issues summarized above, the Committee 
also deliberated on other matters, such as listing potential 
sources of authentic materials and making a list of major Polish 
language programs. A copy of the complete minutes of the 
meting may be found in Appendix G: Minutes of First Local 
Development Committee Meeting. 

2.2.2 Item Writer Traini ng Workshop 

CAL staff designed and conducted a one-and-a-half day 
intensive training program for the two primary item writers. The 
training program offered the item writers an opportunity to 
familiarize themselves with the test specifications and general 
guidelines stipulated by the advisory committees for the project. 
Charles W. Stansfield, Project Director, instructed the group on 
specific issues in relation to structure and reading item 
development. Dorry Mann Kenyon, Associate Project Director, 
supervised the actual practice writing of test items. 

This training enabled the item writers to apply their 
knowledge of Polish and experience in teaching to developing test 
items according to the stipulations set forth for this specific 
project. Since both the item writers were experienced Polish 
teachers, their work and joint revision on items under 
development ensured the good quality of the test items. 

2.2.3 Initial External Advisory Committee Meeting 

Members of the External Advisory Committee were sent copies 
of the Minutes of the Local Advisory Committee Meeting for review 
and comment. in addition, members of the committee met for a 4- 
hour meeting scheduled during the annual conference of the 
American Association of Teachers of Slavic and East European 
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Languages (AATSEEL) in San Francisco, California on December 28, 
1991. The main task of the meeting, besides introducing the ppt 
project to the members and discussing the design of the 
structure items, was to review drafts of the listening and 
reading items. Prior to the meeting, every member was mailed a 
copy of selected items and asked to note down on revision forms 
their individual comments. Thus, every item was reviewed by two 
committee members prior to the meeting. At the meeting, the item 
review process was carried out as follows. First, the two 
individuals assigned with the review of a test item voiced their 
comments and opinions on it. Then, any member of the group had 
the opportunity to comment on that item. In this way the items 
were reviewed one by one. All revisions and relevant comments 
were marked on a single master copy of the items. 

From the item review process and discussions among the 
advisory members, the following concerns and topics of discussion 
emerged : 

1. On the whole, the group felt the test items were very well 
designed. 

2. The group felt that the listening items, as a group, seemed 
to be easier than the reading items. 

3. The group felt that more easier reading items needed to be 
added, and that there should be fewer items taken from 
newspapers. 

4. The group stressed the need to produce a high quality 
recording of the listening passages, using a variety of 
voices. 

5. The members felt comfortable with the idea of trying out 
both the f illing-in-the-blanks and the error recognition 
formats in the structure section, even though they said 
error recognition is not usually done in classrooms. 

The members of the External Advisory Committee reached a 
general consensus regarding the practicality and suitability of 
the test specifications and overall guidelines as recorded in the 
minutes of the Local Development Committee Meeting. They also 



expressed their willingness to offer any assistance needed for 
the project. 
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After the initial start-up of the project, attention was 
focused on the development of PPT test form for field testing. 
This chapter describes the steps in developing the field test 



form 



3 - 1 Initial Develop^*- pf the Test Items 

The two item writers started writing the reading items first 
because of the immediate access to sources of materials. 
Listening and structure items were prepared later, when movies, 
tape recordings and other sources materials were located. From 
October to December, 1991, item writers worked on their own 
schedules and submitted draft items weekly. 

To standardize the item development procedure and to ensure 
the quality of the items, CAL staff designed an item submission 
form to help guide the item writers in the writing process. Item 
writers were instructed to first find authentic listening or 
reading passages suitable for testing comprehension. They were 
then to focus on an appropriate aspect of each passage to test 
comprehension in listening or reading. They then developed the 
question, chose appropriate distractors, and submitted the items 
to CAL for review. 

Each item was submitted on a submission form that required 
item writers to classify and categorize it. A copy of this form 
(one for listening and one for reading) may be found in Appendix 
H: ppT item Submission Forms. 

After items were revised among the item writers themselves, 
they were reviewed by CAL project staff and either accepted, 
discarded or returned to the original item writer for further 
revision. Accepted items were put into the PPT item bank. 
Regular checks were made to ensure that the items being accepted 
were fulfilling the specifications for the whole test in terms of 
their targeted level of difficulty, content (i.e., topic area) 
and speech function. 
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3 * 2 Review of the Test Tt.ems bv th* Rvi-^ r nal Htoimnry commits 

When the item bank contained more items than required by the 
test specifications, items were then sent to the members of the 
External Advisory Committee for further review. Each committee 
member was assigned a subset of the items, all from one section 
of the test. All items were reviewed by two committee members. 
On the basis of input received from the Committee meeting, held 
during the AATSEEL annual conference in San Francisco, items were 
either kept as they were, revised as per specific comments, or 
removed from the item bank. 

The structure items, which were prepared after the meeting 
at AATSEEL, were sent to individual members of the External 
Advisory Committee for review. Their comments were returned to 
CAL and incorporated by the item writers into the items. 

After the draft test form was revised on the basis of input 
from members of the External Review Board, the field test 
materials were prepared. 

3 - 3 Preparing the Fi» M Test Mai-» r i *1<= 

From the items remaining in the item bank, CAL staff put 
together the Field Test Form, following the guidelines set forth 
in the test specifications. Recognizing that items might be 
rejected after field testing, this form contained more items than 
were envisioned for the final form. 

The PPT test booklet for the field testing was prepared 
using WordPerfect 5.1. a special keyboard was designed for 
sections containing Polish text. 

To ensure that the listening passages were all clear and of 
professional quality, they were re-recorded using native speakers 
resident in the Washington, D.C. area. Two male and two female 
voices were selected. The speakers were instructed on how to 
ensure the authentic delivery of the listening passages. The 
speakers strove to read the written script in as natural a way as 
possible, while keeping in mind the need for clear articulation. 
During the recording session at a professional studio, the 
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Associate Project Director served as the director, supervising 
the recording procedure. Two other individuals, one an advanced- 
level student of Polish, the other a Polish language teacher, 
listened critically to each passage as it was recorded. These 
two persons either agreed that each take was of appropriate 
quality for inclusion on the test, or gave input into improving 
the retake. English sections of the Master Tape (test 
instructions and questions on the listening passages) were 
recorded by a professional announcer. The recordings were edited 
by professionals at the recording studio. 

The final test booklet for field testing contained 65 items 
in the Listening Comprehension section, 56 items in the Reading 
Comprehension section, and 40 items in the Structure section. In 
addition to the test booklet and master tape, other materials 
were prepared for the field testing. These included the 
following: 

• A machine-readable standard answer sheet 

• An instruction manual for test administration 

• An examinee background information form 

• An examinee familiarization sheet 

When this process was completed, the PPT was ready to be 
field tested. 
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4. Field Testir^ 



In order to ensure that the developed ppt font was valid and 
appropriate for the group for which it was designed, the PPT was 
field tested between February and April of 1992 on examinees from 
Polish programs in the United States and Canada. This chapter 
describes the field testing procedures and the results. 

4.1 Administration of the Test 

The PPT was designed to be appropriate for students in the 
range from Novice-High to Advanced-High on the ACTFL scale. 
Since language programs do not classify themselves according to 
the ACTFL scale but in terms of years and credits, project staff 
felt that the test should be appropriate for students from the 
end of a first year college level course to students in a third 
year course, it was desired to include examinees at both the 
lower and higher ends of this range in the field testing sample 
in order to compare examinee performance across the entire scale. 

CAL developed a list of all institutions in North America 
that indicated they offered Polish instruction. This was done 
with the help of members of both the local and external 
committees, and through the use of two directories published by 
the American Association for the Advancement of Slavic Studies: 
Directory of Programs in Soviet and East European Studies 1990- 
1992 and Directory of Members 1991-93. Every institution was 
contacted and invited to participate in the field testing, in 
this process CAL encountered two practical problems. The first 
was that a number of schools listed in the directories had 
dropped programs due to budgetary constraints. The second was 
that many of the programs were staffed by part-time instructors 
who were not in a position to manage or coordinate formal 
testing. 

in the end, instructors from 15 different institutions 
participated in the field testing of the PPT. At the University 
of Manitoba (Canada), the instructor administered the PPT to 



students at other institutions, including high school programs. 
A total of 129 examinees participated in the field testing. 
Table 4 indicates the number of examinees completing at least one 
section of the test together with their institution or, in the 
case of the University of Manitoba, that of the test coordinator. 



Name of Tnstitm-i^p 

American U 

Columbia U 

Harvard U 

McMaster U 

Stanford U 

SUNY, Stony Brook 

U of Manitoba 

U of Chicago 

U of Pittsburgh 

U of Minnesota 

U of Massachusetts 

U of North Carolina 

U of Connecticut 

U of Florida 

Villa Marie College 

Total 



Table 4 
Field Test Participants 



State 


Number of 


DC 


4 


NY 


13 


MA 


12 


CANADA 


2 


CA 


6 


NY 


7 


CANADA 


22 


IL 


8 


PA 


3 


MN 


9 


MA 


4 


NC 


11 


CT 


7 


FL 


16 


NY 





129 



Examinees completed a background questionnaire before taking 
the field test version of the PPT (see Appendix I: Examinee 
Background Questionnaire). Percentages used in the following 
description are based on the number of examinees answering each 
question. Fifty percent of the participating examinees were male 
and 50% were female. Ninety percent were at a college or 
university, 5% were in a public high school, and 5% were studying 
Polish at a community school or other institution, sixty percent 
reported that they are ethnic Polish. The majority of all 
examinees (60%) indicated that they spoke no Polish at home. Of 
all the examinees, 13% indicated that they spoke Polish at home 
"most of the time," 10% indicated that they spoke Polish at home 
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"less than half the time," and 17% indicated that they spoke 
Polish at home "very infrequently." 

In terms of the level of study at the time of the test 
administration, the participants were enrolled in classes ranging 
from the third year of a high school course to more than third 
year college level Polish language course. Of those indicating 
their level (117 students), the majority (61%) indicated that 
they were in either the first or second semester of a first year 
college level Polish course. 56; of all the students had not yet 
completed one year (college or high school) of Polish 
instruction, while 12% had completed four or more years. Table 5 
shows the level of study indicated by the field testing 
examinees. 



Table 5 

Levels of Polish Study of the Field Test Examinees 

Level of Polish Instruction 
None of the Options 
3rd Yr High School 
4th Yr High School 
1st Semester or 1st Quarter 
2nd Semester/2nd or 3rd Quarter 
3rd Semester/4th Quarter 
4th Semester/ 5th or 6th Quarter 
5th Semester/7th Quarter 
6th Semester/8th or 9th Quarter 
More than 3rd Year 

Total 

•Missing = 12 
b Due to rounding 



4.2 Results Of the Field Testing 

Both quantitative and qualitative data were collected on the 
field test form. Quantitative data consisted of the examinees' 
responses to the test items. Qualitative data consisted of 
comments made by test supervisors, and, from one school, 



Frequency 

6 

3 

3 
13 
58 

7 
12 

1 

5 

2 

117* 



Percent 
5 
3 
3 

11 

50 
6 
10 
1 
4 
8 

101% b 



examinees themselves. 

Examinees recorded their responses to the background 
questionnaire and their answers to the test on the General 
Purpose Answer sheet form no. 4521 of the National Computer 
Systems (NCS) . Each sheet was scanned on CAL's NCS Sentry 3000 
Optical Scanner twice: the first time to collect background 
information using the program Scantools, the second to score the 
test using software MicroTEST Score II Plus, both by NCS. The 
two databases thus entered were merged into one file using the 
Paradox database system. Item analysis was conducted using the 
Test Analysis Program (TAP) , a classical item and test analysis 
program, and BIGSTEPS, a Rasch model IRT-based item and test 
analysis program. Statistical analysis was performed using SAS. 

Item analysis was first conducted on the entire group of 
examinees. Table 6 presents the summary of descriptive 
statistics by section. 

Table 6 

Descriptive Statistic* fro* the Field Testing 

«*>er Mean Std. Mean Std.Dev. 

* Score Dev. Reliability P-value p-yaluc 

« 58.76 12.80 .93 .60 .19 

56 36.32 11.65 .93 .65 .H 

*0 22.36 9.13 .91 .56 .H 



Table 6 shows that the subtest reliabilities were quite 
high, most likely a function of the length of each part and the 
fact that a wide range of abilities were represented in the 
sample. The mean p-value for the listening and reading items was 
appropriate for a multiple choice test. The reading section was 
the easiest subtest, while the structure section was the most 
difficult. 

The next step was to analyze the individual items to detect 
if any were malfunctioning. Information on the items was 
acquired through TAP and BIGSTEPS. TAP analyses provided point- 
biserial correlations, p-values, frequency of responses to each 
item broken up by quintiles, graphic representations of the 
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Section Exearinees 
Listening 105 
Reading 118 

Structure 127 
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percent of correct responses to test items by quintiles, and the 
number of examinees choosing each option, by quint ile. BIGSTEPS 
provided information on the fit of each item (and person) to an 
underlying Rasch measurement model. Commonly accepted criteria 
were used to identify problematic items: p-values that were below 
30; point-biserial correlations below .30; and Rasch fit 
statistics above 2.0 or below -2.0. 

This analysis revealed that the majority of the items on the 
PPT were functioning very well. The numbers of items that were 
flagged by the above criteria were: 8 of the 65 listening items, 
8 of the 56 reading items, and 3 of the 40 structure items. 

In addition to the statistical analysis, a qualitative 
analysis at the item level was undertaken based on written 
comments received from the test supervisors and from one group of 
students. Moreover, the entire field test form was carefully 
revie ad by one member of the test development committee, whose 
commei ;s were also carefully considered. The vast majority of 
comments were highly favorable. On the item level, most of the 
comments related to items that were flagged through the 
statistical analyses. The information provided by both types of 
data corroborated each other. A few comments had to do with 
personal preferences of the test supervisors. 

4.3 Revision of the T *>*+ 

After both the quantitative and the qualitative information 
was summarized, the local test development committee met to 
discuss the field testing results and what to do about the 
flagged items. since extra items were included in the field test 
form, the committee felt free to delete from the final form any 
items that were technically flawed rather than work on revising 
them. At the meeting, the committee agreed that the final test 
length should be 50 items each for the listening and the reading 
sections. 

In the listening section, 10 items were deleted by the 
committee on the basis of the quantitative and qualitative 
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analyses. Only one flagged item received minor revision that may 
affect its quality: one of its distractors was revised so as not 
to be as plausible as the original distractor was. One other 
superficial change was made to two options in an item in order to 
make all the options parallel. Other than these, no revisions 
were made to any of the listening items. 

In the reading section, three of the eight flagged items 
were easier items with point-biserial correlations slightly below 
.30. These items showed no misfit to the Rasch model. The 
committee decided that these items should not be automatically 
deleted. At this point they were all retained for their content. 
Four clearly malfunctioning items were deleted, however. Minor 
typographical errors were made to five of the items; three 
affecting the Polish text, two affecting the English. The 
committee felt that these minor changes would not affect the 
quality of the items. 

In the structure section, the analysis indicated that three 
of the five items in the last of the eight passages appeared to 
be much too difficult for the field test examinees. Two of these 
items also showed misfit problems. It was decided to delete this 
passage as a whole rather than to repair its items. Only two 
other items showed minor problems, which were easily corrected. 
Thus, the final version of the structure section contains 7 
passages with 5 items each, for a total of 35 items. 

Following the meeting, there were still 5 items from the 
listening section and 2 from the reading section to be deleted in 
order to bring the number of items in each section on the final 
form down to 50. The selection of these items was made through 
the use of the BIGSTEPS program. For each section, after the 
initial run, misfitting persons and the items to be deleted were 
omitted from a second run. The fit of the items was then 
examined a second time, and the remaining number of items to be 
deleted were selected on the basis of this analysis of fit. 

Once the final items were selected, the recording studio 
removed the deleted items from the tape for the listening 
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comprehension section and a revised test booklet was prepared. 
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As reported in Chapter 4, CAL invited every known program 
listed as offering Polish to participate in the field testing of 
the PPT. For various reasons, many programs could not 
participate in organized testing. All programs able to 
participate did so. This meant that no large pool of Polish 
examinees who did not participate in the field testing were 
available on whom to test the final version of the PPT at the end 
of the spring semester of 1992. Thus, in the fall of 1992 the 
final form of the PPT was being administered to over 30 examinees 
from invited government agencies in order to check for minor 
infelicities. After their answer sheets are returned, the final 
version of the PPT Test Booklet will be printed. 

Nevertheless, it was possible to estimate examinee ability 
scores, compute test statistics, and to construct tables for test 
score interpretation from the field test data. This process is 
explained in this chapter. 

5.1 Rationale 

Using irt (item Response Theory), an examinee's ability is 
estimated independently of the items administered. This is 
called -item-free person measurement. « An examinee's score is 
not solely based on the number of correct answers on the test 
but is empirically estimated based on the difficulty level of 'the 
items the examinee answers correctly. Thus, in the context of 
the development of the PPT, an examinee's ability can be 
estimated based on responses to the entire set of items 
administered in the field test, it can also be derived from any 
subset of those items, m general, estimating ability from fewer 
items means a loss in the statistical precision of the ability 
estimate. However, within statistical error, the ability 
estimates based on the entire set or any subset will be the same. 

Given this context, the ability of each examinee 
participating in the field test was estimated using the data from 
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the items remaining on the final form. This data was analyzed 
with the BIGSTEPS computer program, which uses the Rasch irt 
model. This data set, and the ability estimates from the Rasch 
analysis, were then used in constructing the preliminary test 
interpretation tables for the PPT. 

5 - 2 Building the Polish Proficie n cy Test Se a, ] * 

As an aid in interpretation, the PPT Scale for reporting 
scores was developed. After the Rasch analysis, all ppt 
examinees had an ability score on the same scale. This ability 
score is no longer a "number right" score, but an estimate of the 
person's ability along the continuum of the latent trait (the 
construct being measured) reported in terms of logits (a ratio in 
natural log odd units) centered at 0 (the average item 
difficulty) and extending (for the PPT) from about -2.5 to +5. 
An examinee's ability score in logits is defined as the point on 
the item difficulty scale where the examinee has a fifty percent 
chance of getting the answer correct. Thus, an examinee with an 
ability of 1.00 logits has a fifty percent chance of getting an 
item with a difficulty level of 1.00 logits correct. This 
examinee's chances of getting an item at 0.00 logit is greater 
than 50%, while for an item at 2.00 logits it is less. 

This logit scale, however, can be changed by any linear 
transformation without losing its linear quality, one standard 
approach is to give the transformation a meaning in terms of 
norms. For the P.<>T, a "norming sample" was selected that 
consisted of 70 examinees from the 129 who participated in the 
field testing. This was the total number of examinees who 
completed all three section of the PPT and indicated that they 
were studying Polish at the college or university level. This 
sample of 70 examinees is most likely typical of North American 
students studying Polish at the college level in any given year. 
At the time of norming, 67% percent of the group were in their 
first year of Polish study, 24% were in their second, and 9% were 
in their th rd (or higher) year. Forty-eight percent were male 
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and 52% were female. Fifty-eight percent were of Polish 
ethnicity. Sixty-four percent of the group indicated that they 
spoke no Polish at home, while 3% spoke Polish at home "most of 
the time," 6% spoke Polish at home "less than half the time » and 
27% spoke Polish "very infrequently" at home. 

The mean score of this group in logits was assigned the 
easily-remembered scale point of 100. One standard deviation in 
the distribution of the scores of this group was set to b, equal 
to 20 scale points. The PPT Scale score of 100 could then be 
interpreted as the average ability score for the norming sample 
of PPT examinees. An examinee receiving a score of 140 would be 
two standard deviations above this mean; an examinee receiving a 
score of 80 would be one standard deviation below this mean 

Below are the three equations used to transform ability 
estimates in logits to scores on the PPT Scale. Scaled scores 
are always rounded to the nearest integer. 

Listening Compr ehensio n 

Scale Score = 89.90 + (14.27xLogit Score) 
Reading CoBDre^pc^n 

Scale Score = 88.71 + (15.6ixLogit Score) 
Structure 

Scale Score = 96.14 + (15.3ixLogit Score) 

5 * 3 Balding the Test interpret**- t ei M — 

Because of the small number of Polish examinees who could be 
involved in the norming of the PPT, it was impossible to give 
accurate norms at every score level. For this reason, the primary 
test score interpretation table is a table of quartile 
distributions, shown in Table 7. This table provides quartile 
scores on the PPT Scale for examinees in the first, second, or 
third or more year of Polish language study. The figures are 
based on the results of all 129 examinees who participated in the 
field testing of the PPT. However, not every examinee took every 
section of the PPT . T he characteristics of the entire group of 
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129 examinees did not deviate substantially from those of the 70 
examinees who comprised the norm group, described in Section 5 2 
The 70-member norm group was a subset of the entire group of 129 
examinees. 

Quartiles represent the bottom, second, third and top 
quarter of the score distribution. A score in the second quartile 
indicates that the examinee performed in the second quarter of * 
all students in his or her subgroup, and better than the bottom 
25 percent of all the examinees in that subgroup. A score in the 
fourth quartile indicates that the examinee scored in the top 25% 
of all other examinees in the subgroup; i.e., better than 75% of 
all other examinees. 



Tabic 7 

Quartile Distributions for the PPT 







First Year 




Second Year 


Third Year or Higher 


Quartik 


List 


Read 


Str 


List 


Read 


Str 


List 


Read 


_£l 


4th 

(76-100%) 


99 and 
above 


105 and 
above 


106 and 
above 


121 and 
above 


U9 and 
above 


123 and 
above 


139 and 
above 


144 and 
above 


154 and 
above 


3rd 

(51-75%) 


92-98 


93-104 


93-iu5 


105-120 


107-118 


110-122 


130-138 


127-143 


122-153 


2nd 

(26-50%) 


83-91 


83-92 


84-92 


97-104 


99-106 


104-109 


101-129 


109-126 


108-121 


1st 

(1-25%) 


82 and 

below 


82 and 

below 


83 and 
below 


96 and 
below 


98 and 
below 


103 and 
below 


100 and 
below 


108 and 

below 


107 and 
below I 


Examinees 


(56) 


(60) 


(69) 


(18) 


(17) 


(19) 


(6) 


(15) 


(15) | 



To use Table 7 in interpreting an examinee's score, the test 

t n !r d ? S V dentifieS " aBinee " eith « » " r "' —»-. or 
third (or hl ,her> year student. -Then, for each section, the user 

fmds the score group in the table that contains the examinee's 



score and reads the quartile group from the left hand column that 
corresponds to the examinee's score. 

In a few cases, examinees in the norm group indicated that 
Polish was used in the home more than half the time. Such 
examines may have an advantage on a test of general language 
proficiency. However, since these examinees formed only a small 
minority of the total, and since they do comprise a pa^t of the 
total group of North American students studying Polish, they were 
included in Table 7. Nonetheless, PPT users may want to 
understand the meaning of an examinee's test scores in terms of 
whether the examinee indicates that much Polish is spoken at home 
or not. Table 8 presents the average PPT Scale score for each 
section by year of study for examinees who indicated that they do 
not generally speak Polish at home (labeled English) and those 
who indicated that Polish is spoken about half the time or more 
(labeled Polish). Underneath each average score is the number of 
examinees upon which the mean is based. The more examinees, the 
more reliable the mean score is. Means for subgroups with less 
than 3 members are not shown. 



Table 8 

Mean Scores on the PPT Scale 
by Level and Home Language 
(Number of Examinees in Parentheses) 





Listening 


Reading 


Structure 


Year of Study 


English 


Polish 


English 


Polish 


English 


Polish 


First Year 


92 
(53) 


(-) 


93 
(55) 


96 
(5) 


94 
(64) 


96 
(5) 


Second Year 


107 
(15) 


(-) 


107 
(15) 


(-) 


110 
(16) 


... 

(-) 


Third Year 
(or higher) 


124 

(5) 


(-) 


119 

(7) 


130 
(8) 


121 
(7) 


129 
(8) 



To use Table 8, the test users finds the group to which the 
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examinee belongs and compares his or her score to that group's 
mean scot e . 

The PPT is intended for English-speaking learners of Polish. 
Since the questions and options for the Listening and Reading 
sections as well as the directions for the entire test are in 
English, knowledge of English is required to take the test. Thus, 
it is impossible to use these tests to measure the Polish 
language proficiency of native Polish speakers (or speakers of a 
language other than Englxsh) who do not have a good knowledge of 
English. Test results may not be accurate for any examinee who 
does not have a good command of English; i.e., a level of English 
language proficiency generally equivalent to that of a first-year 
college student enrolled in the regular program of an American 
university. 

In the field testing of the PPT, 17 of the 129 examinees 
indicated that they were of Polish ethnicity and spoke Polish at 
home -most of the time." The responses of these examinees were 
carefully examined to see to what extent, if any, their 
proficiency in the English language may have been affecting their 
performance on the test, one analysis involved checking to see 
if these examinees were getting easier items incorrect; i.e. 
easy items that they may have missed because of their restricted 
ability in English rather than their proficiency in Polish. None 
of the responses of the Polish speakers showed any incongruities 
on any of the three sections of the test. On the basis of 
experience to date, it is reasonable to assume that the level of 
English used in the test will not pose a problem to the native 
Polish speaking examinee who has enough proficiency in English to 
be enrolled at an English-speaking university in North America. 

The information contained in this chapter and the following 
is also presented in a more -user-friendly" manner in the PPT 
Test Interpretatio n Manual . 



6 - Psychometric Prope rties of the Test 



This chapter discusses the technical aspects of the PPT. it 
must be remembered that all the statistical analysis are based on 
the results from the field testing. However, only performance 
data on t>~e items retained in the final form of the PPT were 
used. 

6.1 Test Reliability 

The reliability of a test is the extent to which it yields 
consistent results. Thus, high test reliability is desirable. 
Table 9 gives the Rasch equivalent of the KR-20 reliability of 
each section of the final form of the PPT based on the data from 
examinees involved from the field testing. The number of 
examinees for each section is: Listening (96), Reading (110), and 
Structure (123) . 



Table 9 
Reliability of the 
PPT by Section 

? e<: *i on Reliability 
Listening Comprehension .92 
Reading Comprehension .90 

Structure [ 88 



The figures presented in Table 9 indicate that the PPT is a 
highly reliable tests which can be used with confidence by 
programs needing trustworthy measures of Polish language 
proficiency. 

6.2 Measure ment Pre ^jRjrm 

Any measurement involves a degree of error. The higher the 
precision of the measurement, the smaller the error. In Classical 
Test Theory, the degree of error is usually indicated by the 
standard error of measurement (SEM) . One of the limitations of 
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this approach is that it represents the SEM as being the same for 
all examinees. Test scores, however, are unequally precise for 
examinees at different levels of ability. Item Response Theory 
(IRT) , used in scoring the PPT, also provides a measure of the 
precision of each score on the PPT Scale. In IRT, this measure is 
called the standard error of the ability estimate. Appendix J: 
Table of Standard Error of Estimation (also contained in the ppt 
Test Manua l) contains the standard errors for PPT scores, rounded 
to the nearest whole number. 

A traditional SEM may also be calculated for the PPT on the 
PPT Scale. These are presented in Table 10. Figures have been 
rounded to the nearest whole number. 



Table 10 

Standard Error of Measurement (SEM) 
for the PPT 
(in Rounded Scale Score Units) 

Section SEM 
Listening e 
Reading 7 
Structure 7 

Both the SEM from Table 10 and the standard error of the 
estimate from Appendix J give an indication of how accurate a 
single test score is. Either can be used to construct confidence 
intervals around PPT scaled scores. The confidence interval takes 
measurement error into account. One way to view this confidence 
interval is to say that the examinee's true score (i.e., the 
average score an examinee would receive if it were possible to 
give him or her the test a very large number of times) would lie, 
with 67* certainty, within plus one or minus one standard error 
of the given score. Another way to view this is to say that on a 
re-test, if there has been no change in the examinee's true 
proficiency, the examinee will score within plus or minus one 
standard error of his or her score about 67% of the time. 

A careful examination of Appendix J reveals that the most 
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precise area of measurement on the PPT lies between 66 and 136 on 
the PPT scale. For scores above 136, the measurement precision of 
the test rapidly diminishes. A score of 136 indicates a 
performance well above the cut-off point for the fourth quartile 
for first and second year students, and in the third quartile for 
students in the category -third year and above." In other words, 
the measurement precision of the PPT should not be of concern in 
interpreting the scores of first and second year Polish students 
on any section of the test. However, test score users should be 
aware that the PPT may not be able to discriminate accurately 
between very high scoring examinees on the PPT; i.e., examinees 
who score above 135 on any section. Table 8 indicates that such 
examinees would generally be individuals who have completed three 
or more years of Polish study and who speak Polish at home. 

6.3 Validity of the Test 

Validity refers to the extent to which a test actually 
measures what it purports to measure. The PPT claims to measure 
an examinee's proficiency in understanding authentic spoken and 

| written Polish, and ability to deal with Polish structure. 

' The validity of any test cannot be "proven;" it can only be 

established by the collection of evidence that the test is indeed 
measuring what it purports to measure. Some commonly accepted 
types of evidence include evidence for content, concurrent, and 
construct validity. Each type of validity, and evidence which 
supports it in the case of the and the PPT, are explained below. 

Content Validity 

Content validity is based on test content and refers to the 
extent to which the items in a test adequately sample the 
linguistic and behavioral domains of the construct that the test 
is meant to measure. The PPT is intended to be a measure of 
proficiency in dealing with every-day "real-life" Polish. 
Validity based on content thus entails an examination of the 
degree to which the tests, in their stimulus passages, sample 
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from the corresponding language-use situations the examinee might 
be expected to encounter in real life. 

1 . Content Domains 

The ACTFL Guidelines for listening and reading postulate 
certain content and functional domains as appropriate to the 
proficiency of an English-speaking learner of Polish at different 
levels of proficiency. The developers of the PPT used authentic 
listening and reading passages and wrote corresponding items to 
reflect these domains. Content and functional domains can be 
grouped into three categories, which represent increasing levels 
of proficiency in Polish as a second language. These levels may 
be designated as: 

intermediate Level— survival. At this level, the learner can 
"survive- by performing simple daily tasks such as shopping 
for food, clothes, and incidental necessities. PPT listening 
comprehension items at this level require the understanding 
of oral language typically heard in locations such as a 
grocery store, train station, bank, bus, and airport. PPT 
reading comprehension items at this level include certain 
signs, advertisements, storefront designations, simple 
messages with a social purpose, and short straightforward 
instructions encountered in daily life. 

?!3S C *?H^7* 1 ~ TOUri,t in Polish Society. At this 

level, the learner can understand social conversation on 

p£? r ??o£ t ° P1CS and Can read short ' descriptive narrations. 

listening comprehension items at this level require 
comprehension of short conversations between two people 
o? a ii2L™ general, every day topics, or the comprehension 
of passages such as news items and reports dealing primarily 
?h?5 ?oSo? al ln ? ormation - p PT reading comprehension items at 
«™ I t 1 " guire the examinee to comprehend main ideas and 
some facts from passages excerpted from newspaper articles 
storift S n ° tlCeS ' personal correspondence, and popular 

Superior Level— Foreign Professional in Polish society. At 

this level, the learner can comprehend oral and written 
language on virtually any matter of general interest to 
educated native speakers in authentic Polish situations. PPT 
listening stimuli at this level involve conversations 
between two speakers dealing with topics of a general 
academic nature such as recent advances in health care and 
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current events, or longer discourse excerpted from speeches 
or reports. Questions nay deal with the speaker's opinion or 
attitude, or they may require inference on the part of the 
examinee. Reading stimuli at this level are generally longer 
and more technical and are followed by a questions dealing 
with basic factual information covered in the passage, or 
inference or analogy based on information presented in the 
passage. The passages cover a wide variety of daily topics 
such as those that may be found in a newspaper or magazine, 
e.g., results of a public opinion survey, job descriptions, 
and vacation travel. 

For the Structure section, the PPT tests knowledge of 
correct Polish syntax. Unlike listening or reading comprehension, 
the t^ructure section does not involve real-life language-use 
Vdsks per se, though clearly knowledge of Polish syntax is a part 
of comprehending spoken and written Polish. Items testing 
recognition of Polish syntax on the PPT are of two types. The 
first typ.* 1 asts knowledge of correct structure by requiring the 
examinee to choose from among four options the correct word or 
phic that should be inserted in a blank within a printed 
paragraph. The second type tests structural knowledge by 
requiring the examinee to find from among four underlined words 
the one that is grammatically incorrect. In this section of the 
PPT, only Polish is used for both the stimulus and response 
options. 

Table 2 of this report contains a list of the topic areas 
covered in the PPT listening and reading comprehension sections. 
Table 3 contains a list of speaking functions covered on the 
listening section of the test. A list of the comprehension tasks 
on the listening and reading comprehension sections is contained 
in Appendix K: Comprehension Tasks for the PPT Items. A list of 
grammar topics (by part of speech) covered in the structure 
section can be found in Appendix L: Grammatical Areas Covered in 
the PPT Structure Items. 

2. Authentic Sources 

Stimulus passages for listening, reading and structure on 



the PPT were drawn from authentic language sources, item 
developers collected passages for the reading comprehension and 
structure sections from sources such as Polish language 
newspapers, magazines, journals, books, street signs, schedules, 
and tickets. For stimulus passages for listening comprehension, 
they listened to and transcribed listening passages from sources 
such as Polish language news broadcasts, movies, and 
announcements. These texts were modified only to the extent that 
they needed to be clarified when taken out of the larger context. 
Questions were designed to check comprehension of the meaning of 
passages, and, for more difficult passages on the PPT, to check 
comprehension of opinions, attitudes or inferences contained in 
the passages. A representative list of sources for the listening 
and reading passages used on the PPT is contained in Appendix M: 
Sources for PPT Passages. 

In terms of content validity, then, it can be demonstrated 
that the PPT items have been drawn from real-life use of Polish 
language. 

Concurrent Validity 

A second type of evidence of validity is concurrent 
validity. Concurrent validity refers to the extent to which a 
test score correlates with results that may be obtained through 
the use of independent criteria external to the test, measured at 
the same point in time, to see if expected relationships exist. 

For the PPT, the reported level of Polish study may be used 
as an external criteria, it would be expected that scores on the 
tests would increase as amount of study increases. Both Tables 7 
and 8 indicate that this is the case for the PPT. In each skill, 
the median (Table 7) and average (Table 8) score increases as the 
level of Polish instruction increases. These tables thus provide 
evidence of the concurrent validity of the PPT as a measure of 
Polish language proficiency. 
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Construct Validity 

A third way of examining validity is construct validity. The 
goal of construct validity is to determine whether or not a test 
measures a single underlying trait. One assumption of the IRT 
model used to develop and score the PPT is that the items are 
"unidimensional"; that is, only one examinee ability or trait is 
necessary to account for performance on the test. The fit 
statistics provided by the IRT analysis provide evidence of the 
extent to which unidimensionality exists. Although perfect fit is 
impossible, it may be argued that if the overwhelming majority of 
items are appropriately fitting, then there is strong evidence 
for the construct validity of the test. For each item, a 
standardized outfit statistic is calculated. One common criterion 
is to consider any item for which this statistic is above 2.00 or 
below -2.00 as misfitting. Using this criterion, only 3 of the 50 
listening items (6%) and 2 of the 50 reading items (4%) on the 
final form were misfitting. This high degree of fit provides 
strong evidence for the construct validity of these measures, in 
the structure section, 5 of the 35 items (14%) were misfitting. 
Although 14% represents a higher proportion of misfitting items 
than for the listening and reading sections, the figure is low 
enough to lend some support the construct validity for the 
structure section. It may be noted that one of these five 
misfitting items has been revised in the final form. 

Validity Summary 

The above discussion provides evidence for the validity of 
the PPT. CAL plans to continue research pertinent to the validity 
of the test and will update the validity information section in 
the PPT Test Interpretation n„ nMfl as appropriate. 

6 - 4 Intercorrelations * p ona r^t g ^ fr g? ^ r?r 

The three sections of the PPT are designed to measure 
different skills within the general domain of Polish proficiency. 
It is expected that these skills are interrelated; i.e., persons 
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who are highly proficient in one skill area will tend to be 
proficient in the other skills as well. However, the 
intercorrelations are not expected to be perfect. If they were, 
there would be no need to report scores for each section; the 
subscores would represent the sane rather than different aspects 
of language proficiency. 

Table n shows the Pearson product -moment correlation 
coefficients measuring the extent of relationships among the 
three subsections of the PPT based on scores on the final form of 
the PPT for the 86 examinees who took all three sections in the 
field testing. 



Table 11 

Intercorrelations Among Subscores 

List Read Str 

Listening 

Reading .84 

Structure .82 .92 



Table 11 indicates that the.o is a fairly strong relationship 
among the skills tested by the three subsections of the test. 
This relationship is strongest between reading and structure 
(.92), which may be expected in a highly inflected language as 
Polish. Nevertheless, it is reasonable to say that each 
subsection scores provides some unique information about the 
examinee's proficiency in the Polish language. This pattern of 
intercorrelations between the test scores helps support the 
validity of the constructs the PPT claims to measure. 
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7. Qperationalization of the ppt 



Throughout the project, CAL staff sought to have the greatest 
amount of input possible from the Polish language teaching 
community and to actively disseminate information about the 
project. Towards the end of the project, the project staff 
created all the necessary materials to ensure that the PPT could 
be appropriately operationalized; i.e., that any appropriate 
individual may administer the test to examinees in his or her 
program in a smooth and timely manner, that the test 
administrator would have information on how to interpret test 
scores, and that steps to safeguard the security of the test 
would be in place. These materials were described in Section 
1.5. This chapter describes some additional steps to disseminate 
information on the PPT program. 

1 ' 1 Information Disseminata Purina the Pro-iecl- 

During the test development phase, efforts were made to inform 
the Polish language teaching community of the project, and in 
many cases allowed for their input. A general announcement on 
the project was developed in response to requests for 
information. A copy this announcement is supplied as Appendix N: 
PPT Announcement. In addition, CAL staff made several conference 
presentations. An outline of dissemination activities to date 
follows. 



B. 



The Modern lan guage Jo urnal 

An announcement about the project was sent to the News 
and Notes Editor of the Modern Language Journal for 
publication. 

Lanauaaes A SSOCiati ° n ° f Slavin and East " F"r"™ a P 

S~ Xixiang Jiana * the Project Coordinator, delivered a 
presentation on the PPT development project at the 
5i»-?n EL ,SI!? Ual conference in San Francisco, December 



Mr. Jiang also presented on the completed project at 
the annual AATSEEL conference in New York on December 
28, 1992. 

c - ggprqe Washington University c olloquium on Foreign 

Language Tes<- fn^ 

Project staff Dorry Mann Kenyon, Rafal Kwiatkovski, and 
5w r ™ W * ?tansfield presented an hour presentation on 
the PPT entitled "Testing reading and listening 
proficiency in Polish" on April 10, 1992, at George 
Washington University in Washington, DC. 

D * 5Pth Anniversary Internation a l Congress of the Polish 

institute of Art^and Scienc es of Ami»rT^ ^*mu 

Mr. Xixiang Jiang, delivered a presentation on PPT 
Prefect in a session entitled "Teaching Polish Language 
and Culture" at the annual conference in New Haven 
Connecticut, June 18-20, 1992. ' 

7 ' 2 Information Dissemination Foi loving Project Comp i erjpn 

Several steps will be taken to disseminate information about 
the PPT upon project completion, including the following. An 
informational Flyer will be developed and mailed to Polish 
programs across the country and to members of the American 
Association of Teachers of Slavic and East European Languages. 
In addition, copies of this final report will be made available 
to the public through the ERIC Clearinghouse. Finally, CAL staff 
will be available to present on the project at relevant venues. 
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Appendix A 
PPT Test Booklet 



POLISH PROFICIENCY TEST 



General Directions 

This is a test of your ability to understand spoken and written Polish. There are three 
sections in this test: Listening Comprehension, Reading Comprehension and Structure. 

In the Listening Comprehension section, you will hear and answer questions about a 
number of spoken passages in Polish, including single sentences, conversations, radio 
broadcasts, and other types of passages. In the Reading Comprehension section, you will 
read and answer questions about a variety of printed materials ranging from one or two 
words to texts of paragraph length. In the Structure section, you will be asked to complete 
the missing portion of paragraphs written in Polish, and identify Polish words within 
paragraphs that must be changed in order for the text to be correct. The test is intentionally 
challenging, and you are not expected to know the answer to every question. 

In each of the three sections of the test, the questions are printed in the test booklet. You 
are to mark your answer to each question on the separate answer sheet provided. Make no 
marks on the test booklet itself. 



Your score on the test will be based on the number of questions you answer correctly. 
Throughout the test, it will be to your advantage to answer every question, even though 
you may not be sure your answer is correct. There is no penalty for guessing. 

When marking your answers, be sure that your mark is dark and completely fills the 
intended answer space. If you make a mistake or wish to change an answer, erase your 
first answer completely. Questions for which more than one answer is marked will receive 
no credit. 



The Listening Comprehension and Structure sections are each divided into several parts 
Additional directions are given at the beginning of each part. 



DO NOT OPEN THE TEST BOOKLET UNTIL YOU ARE TOLD TO DO SO. 
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Appendix B 
PPT Answer Sheet 
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IMPORTANT INFORMATION ON TEST ADMINISTRATION 



lest Security 

It is extremely important that the PPT be safeguarded and administered under secure conditions at 
each participating institution. By agreeing to serve as the supervisor of the test, you assume the 
responsibility of enforcing test security from the time you receive the test materials until you return 
them to CAL. It is, therefore, requested that you adhere to the following conditions: 

1 . Keep all test materials either in your immediate physical possession or in a locked cabinet or 
other secure area under your control. 

2. Do not copy, or allow others to copy, any portion of the test booklet or tape, or make any notes 
or transcriptions of the test booklet or tape content. 

3. Allow only those individuals who are to be tested to see the test materials, and only at the time 
of test administration and under the specific procedures described in these instructions. 

4. Should any irregularities occur, report them in the Supervisor's Report Form included in the test 
package. 

Your serving as test supervisor constitutes an implicit agreement to conform with the above 
conditions. Your cooperation in this important matter is greatly appreciated. 



Communic ation with CAL 

If it is necessary to contact CAL by telephone, please call (202) 429-9292 and ask for the PPT 
coordinator. Correspondence by mail should be addressed to: 

Coordinator, Polish Proficiency Test 
Center for Applied Linguistics 
1118 22nd Street, NW 
Washington, DC 20037 
FAX: (202) 659-5641 



Length of Administration 

The PPT may be administered in either a one-day session lasting approximately 2 1/2 hours or in 
two sessions lasting about 1 1/2 hours on the first day and an hour on the second day. The times for 
each section is as follows: about 40 minutes for the Listening Comprehension Section, 50 minutes for 
the Reading Comprehension Section and 30 minutes for the optional Structure Section. A period of 10 
minutes is also needed for examinees to fill in the Personal Information section of the machine-scoreable 
answer sheet. Please note that it is extremely important that this background information is completed 
correctly. 
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Programs placing a high emphasis on developing proficiency and/or decreased importance on 
grammar may decide to omit the Structure section of the PPT. All scores and norming tables are based 
solely on section scores, so that ommitting the Structure section will in no way hinder the 
interpretability or usefulness of test scores. If the Structure section is omitted, the required test 
administration time is decreased by about 30 minutes. 

Order nf Artmink>r»f inn 

In a one-day administration, the PPT will be administered in the order of the PPT test booklet: 
Listening. Reading and Structure (if included). In a two-day administration, the Listening and Structure 
(if included) sections are administered on the first day, with the Reading section administered on the 
second. These options may be diagrammed as followed: 

One-day Administration: Listening -- Reading - (Structure) 

Two-day Administration: Day One: Listening - (Structure) 

Day Two: Reading 

For the two-day administration, supervisors must collect and store test booklets and answer sheets 
after the first day and redistribute them on the second. 

Test Center Codes 

Each participating institution is assigned a 3-digit test center code for data processing purposes. You 
can find your test center code on the top of the Test Materials Checklist enclosed in the package. You 
will need this information when instructing the examinees to fill out their answer sheets. 
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PRIOR TO THE TESTING DATE 



Receipt of Test Materials 

Immediately after you receive the shipment of test materials, verify the contents against the yellow 
Test Materials Checklist. If you find any discrepancy, you should contact CAL immediately for 
replacement of materials. 

After receiving the test materials, be sure to place them in a securely locked area. Do not allow 
anyone but your designated associate or a responsible official to have access to the storage area. 

Checking the Conditions of Test Materials 

Although every effort is made to ensure that test materials are in good condition before they are 
shipped, it is still possible that some defects may escape our attention or may occur during mailing. It 
is therefore important that you check the condition of the test booklets and test tapes. The following 
steps are recommended: 

1. Check to make sure that all test booklets are properly sealed. 

2. Spot-check the test tapes by listening to random segments all the way to the end to make sure 
that there are no noticeable problems. 

Distributi on of Examinee Handbooks 

In order to prepare the examinees for taking the PPT, it is important that you give each examinee 
a copy of the Examinee Handbook before the testing date. The Examinee Handbook provides 
information on the test and supplies some sample and practice test items. You should receive enough 
copies for your examinees with your order for the test. You may photocopy the Examinee Handbook 
for other interested individuals if you wish, but all other test materials must be kept confidential. 

Arrangements for a Testing SH«» 

You should locate a testing site that is comfortable and free from distraction. The listening 
component of the PPT requires a quiet room with good acoustics throughout and a high-quality cassette 
playback unit. The small speakers of personal battery-powered cassette recorders are not satisfactory 
for this purpose. The test can be administered in a language laboratory if there is enough room in the 
individual booths for the students to place their test booklets and answer sheets flat in front of them. 
The testing room should be large enough so that examinees can be seated with three feet of space in all 
directions between all examinees. Make sure that you will have access to the testing site for the TOTAL 
AMOUNT of time you need it. 
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ON THE TESTING DATE 



Materials Needed for Test AHminktratinn 

In addition to the test materials sent to you by CAL, you will need the following items for 
administering the test: 

1 Playback equipment for the Listening Comprehension section. Check your playback equipment 
to make sure that it is functioning properly. Adjust the volume control so that everybody in the 
room can hear the recording clearly. If the playback unit has a tone control, it should be set to 
the middle ("flat response") position or adjusted somewhat toward the treble. It should not be 
turned toward the bass position. Make sure that the tape is completely rewound afterwards. Two 
identical tapes are provided; you can use either one. 

2. A timer, wrist watch, or other timepiece capable of being reset during the administration for 
timing the Reading Comprehension section. Check to make sure that it is functioning properly 
and has been completely reset to zero (or 12:00). There should always be at least two timepieces 
in the testing room as a check against mistiming. 

3. A few medium-soft (No. 2) black lead pencils with erasers. Examinees will have been asked to 
bring their own pencils, but some may have forgotten to do so. Normally, an amount of pencils 
equivalent to 20% of the total number of examinees will be sufficient. 

Admission to the Testing Rnnpi 

About ten minutes before the testing starts you can start admitting the examinees into the testing 
room. If an examinee shows up who did not register with you ahead of time, it is at your discretion to 
decide whether you wish to admit that examinee. Only those who have paid the $20 testing fee can be 
allowed to take the test. 

All examinees must present proper identification (a photo ID such as a driver's license, student ID, 
or passport). You must verify that the person who registered for the test is indeed the same individual 
who takes the test. If an examinee forgets to bring any identification, you should not admit him or her 
into the testing room. There can be no exceptions to this rule, even if you think you know the examinee 
very well. 

Seating the Examinees 

When seating the examinees, direct them to seats at random to avoid any possibility of pre-arranged 
cheating. They should not be allowed to select their own seats. If at all possible, allow at least one seat 
between any two examinees. 

Prohibited Materials 

While taking the test, examinees should have nothing on their desks but their pencils test booklets 
and answer sheets. The use of books, dictionaries, personal recording devices, photographic devices, 
pens, notes, and papers of any kind is prohibited during the test. 
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ADMINISTERING THE TEST 



Follow the procedures below when administering the test. ALL INSTRUCTIONS PRINTED IN 
ITALICS SHOULD BE READ VERBATIM. Do not depart from these directions unless noted 
otherwise. 



Distributing Test Materials 

After all examinees have been seated, start distributing the following materials: 

1 . Standard answer sheets 

2. Test booklets 

Give the following instructions after the above materials have been distributed: 

Please do not open your test booklet. For this test, you are going to give all your 
answers on the answer sheet. Do not write anything in the test booklet. You must use a 
No. 2 pencil for marking your answers. If you do not have one, please raise your hand. 

Distribute pencils as necessary. 



Filling Out the Answer Sheet 

Instruct the examinees how to answer the questions by filling out the standardized answer sheet. 
(Depending on the examinees' degree of familiarity with standardized answer sheets, you may opt to 
simplify the instructions, or you may want to give some examples on ths board.) 

This is a machine-scoreable test. You must mark your ans\9ers by completely filling in 
the circle provided; otherwise the machine will not score your test correctly. 

All your answers must be put on the answer sheet. Do not mark anything on the test 
booklet; otherwise, your test score may be cancelled. 

This answer sheet has a total of 135 items. The front page has 50 items and the back 
page has the rest. [Hold up the answer sheet and flip over the two sides.] 

Look at side one. In the space marked LAST NAME, print each letter of your last name, and 
darken the corresponding circle below. [Pause] 

In the space marked FIRST NAME, print each letter of your name, and darken the corresponding 
circle below. [Pause] 

In the space marked MI, print the first letter of your middle name, and darken the corresponding 
circle below. [Pause] 
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In the space marked TEST DATE, darken the circle that says [give the current month], then 
write today's day and year, and darken the circles below. [Pause] 

In the space marked SOCIAL SECURITY NUMBER, write your social security number and 
darken the circles below. If you do not haw a social security number, please leave this space 
blank. [Pause] 

In the space marked BIRTH YEAR, write the year you were born, and darken the circles below. 
[Pause] 

In the space marked CENTER CODE, write the number [see the Test Materials Checklist for 
your test center code] and darken the circles below. [Pause] 

In the space marked TEST FORM, write m ll m and darken the circles below. [Pause] 

Next, on the top right hand side of the page, you are asked to answer a few questions about 
yourself and your Polish studies. This information will not affect your score in any way, nor will 
it be reported to anyone without eliminating your name. It is used to describe the group of 
examinees that takes the PPT. Please fill in this information now. [Wait until all have finished] 

Now turn to side two. In the box labelled SIGNATURE, sign your name. In the box labelled 
TEST BOOKLET NUMBER, print the five-digit number printed on the top right corner of your 
test booklet. [Wait until all have finished] 



Are there any questions? [Wait for response] 



(W-day ^minkfr^jftp 



With Structure Section 

Administering the 
listening Comprehension Section 

[NOTE: This section is always administered 
FIRST] 

Prepare the examinees to begin the test by 
saying: 

This test consists of three sections: 
Listening Comprehension, Reading 
Comprehension, and Structure. The total 
testing time is about two hours and ten 
minutes. There will be a short break 
between the sections. 

There are 50 questions in the 
Listening Comprehension section of 
this test. Your answers should be 
recorded on Side 1 of the answer 
sheet. Always make sure you mark 
your answer in the correct space on 
your answer sheet. 

We are now ready to begin the test. 
Please use the end of your pencil with the 
eraser to break the seal on the test 
booklet, but do not open the booklet. The 
Listening Comprehension will be 
administered by a tape recording. All 
necessary instructions are on the 
recording. If you cannot clearly hear the 
recording f please raise your hand. 
Otherwise 1 will not stop the recording 
until it has come to the end. Any 
questions? [Wait for response] 

Now I will start the tape. 

Begin playing the test tape. The tape will 
stan with the words, "This is the Polish 
Proficiency Test/ While these directions are 
being read, walk about the room to make 
sure that the tape is clearly audible to all 



Without Structure Section 

Administering the 
listening Comprehension Section 

[NOTE: This section is always administered 
FIRST] 

Prepare the examinees to begin the test by 
saying: 

This test consists of two sections: 
Listening Comprehension and Reading 
Comprehension. The total testing time is 
about an hour and a half There will be a 
short break between the sections. 



There are 50 questions in the 
Listening Comprehension section of 
this test. Your answers should be 
recorded on Side 1 of the answer 
sheet. Always make sure you mark 
your answer in the correct space on 
your answer sheet. 

We are now ready to begin the test. 
Please use the end of your pencil with 
the eraser to break the seal on the test 
booklet, but do not open the booklet. 
The Listening Comprehension will be 
administered by a tape recording. All 
necessary instructions are on the 
recording. If you cannot clearly hear 
the recording, please raise your hand. 
Otherwise 1 will not stop the recording 
until it has come to the end. Any 
questions? {Wait for response] 

Now I will start the tape. 

Begin playing the test tape. The tape will 
stan with the words, "This is the Polish 
Proficiency Test/ While these directions are 
being read, walk about the room to make 
sure that the tape is clearly audible to all 
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examinees and make any necessary 
adjustments. The total playing time is 
approximately 40 minutes. Please do not 
leave the testing room during this time. 
During the playing of the test tape, it would 
be advisable to walk quietly around the room 
to make sure that the playback volume is 
satisfactory and that the examinees are filling 
in their answer sheet properly. 

At the end of the tape, the announcer will 
say: "This is the end of the Listening 
Comprehension section. Please close your 
test booklet and wait for further 
instructions." You should now stop the tape 
and say: 

We will now have a short rest break. 
Please insert your answer sheet in 
your test booklet and close your 
booklet. We will begin the next section 
of the test in three minutes. [You may 
leave the room if you wish J 



examinees and make any necessary 
adjustments. The total playing time is 
approximately 40 minutes. Please do not 
leave the testing room during this time. 
During the playing of the test tape, it would 
be advisable to walk quietly around the room 
to make sure that the playback volume is 
satisfactory and that the examinees are filling 
in their answer sheet properly. 

At the end of the tape, the announcer will 
say: This is the end of the Listening 
Comprehension section. Please close your 
test booklet and wait for further 
instructions." You should now stop the tape 
and say: 

We will now have a short rest break. 
Please insert your answer sheet in 
your test booklet and close your 
booklet. We will begin the next section 
of the test in three minutes. [You may 
leave the room if you wish.] 



Administering the Reading Comprehension 
Section 

Begin the Reading Comprehension section of 
the test by saying: 

There are 50 questions in the Reading 
Comprehension section of this test. Your 
answers should be recorded on Side 2 of 
the answer sheet, beginning with question 
57. Always make sure you mark your 
answer in the correct space on your 
answer sheet. 

We will now start the Reading 
Comprehension section. Please turn to 
page 9 in your test booklet and find 
'Section II - Reading Comprehension. ' 
There is a total working time of 50 
minutes for this section. You may begin 
working on this section as soon as you 
have finished reading the directions. 

[START TIMER IMMEDIATELY] 



Administering tbe Reading Comprehension 
Section 

Begin the Reading Comprehension section of 
the test by saying: 

There are 50 questions in the Reading 
Comprehension section of this test. Your 
answers should be recorded on Side 2 of 
the answer sheet, beginning with question 
51. Always make sure you mark your 
answer in the correct space on your 
answer sheet. 

We will now start the Reading 
Comprehension section. Please turn to 
page 9 in your test booklet and find 
"Section 11 - Reading Comprehension. m 
There is a total working time of 50 
minutes for this section. You may begin 
working on this section as soon as you 
have finished reading the directions. 

[START TIMER IMMEDIATELY] 
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After 40 minutes, inform examinees of the 
time remaining [Do NOT stop and reset the 
timer]: 

There are 10 minutes remaining in this 
section. 

After 45 minutes, inform examinees: 

There are 5 minutes remaining in this 
section. 

After 50 minutes, inform examinees: 

This is the end of this section of the test. 
Please stop working now. We will now 
have a short rest break. Please insert 
your answer sheet in your test booklet 
and close your booklet. We will begin the 
next section of the test in three minutes. 
[You may leave the room if you wish.J 



Administering the Structure 
Section 

Begin the Structure section of the test by 
saying: 

There are 35 questions in the Structure 
section of this test. Your answers should 
be recorded on Side 2 of the answer 
sheet, beginning with question 101. 
Always make sure you mark your answer 
in the correct space on your answer 
sheet. 

We will now start the Structure section of 
this test. Please turn to page 25 in your 
test booklet and find 'Section III - 
Structure. m There is a total working time 
of 30 minutes for this section. You may 
begin working on this section as soon as 
you have finished reading the directions. 

(START TIMER IMMEDIATELY] 



After 40 minutes, inform examinees of the 
time remaining [Do NOT stop and reset the 
timer]: 

There are 10 minutes remaining in this 
section. 

After 45 minutes, inform examinees: 

There are 5 minutes remaining in this 
section. 

After 50 minutes, inform examinees: 

This is the end of the test. Please stop 
working now. Look over your answer 
sheet carefully. Be sure all the marks 
you made are dark and heavy. Erase 
any marks on your test booklet. 
[Pause] 

Please insert your answer sheet in 
your test booklet and close your 
booklet. Do not leave the room until 
you are told to do so. 



Finishing the Test 

Immediately collect the test booklet from 
each examinee, making sure that the answer 
sheet has been inserted. Be sure to account 
for all test booklets distributed. When all test 
booklets and answer sheets have been 
accounted for, the examinees may be 
dismissed. 
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After 25 minutes, inform examinees of the 
time remaining [Do NOT stop and reset the 
timer]: 

There are 5 minutes remaining in this 
section of the test. 

After 30 minutes, inform examinees: 

This is the end of test. Please stop 
working now. Look over your answer 
sheet carefully. Be sure all the marks 
you made are dark and heavy. Erase 
any marks on your test booklet. 
[Pause] 

Please insert your answer sheet in 
your test booklet and close your 
booklet. Do not leave this room until 
you are told to do so. 



Finishing the Test 

Immediately collect the test booklet from 
each examinee, making sure that the answer 
sheet has been inserted. Be sure to account 
for all test booklets distributed. When all test 
booklets and answer sheets have been 
accounted for, the examinees may be 
dismissed. 
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With Structure Section 

DAY ONE 

Administering the Listening 
Comprehension Section 

[NOTE: This section is always administered 
FIRST] 

Prepare the examinees to begin the test by 
saying: 

This test consists of three sections: 
Listening Comprehension , Reading 
Comprehension, and Structure. Today 
you will take Listening Comprehension 
Section and the Structure Section. The 
total testing time is about 70 minutes. 
There will be a short break between 
the sections. 

There are 50 questions in the 
Listening Comprehension section of 
this test. Your answers should be 
recorded on Side 1 of the answer 
sheet. Always make sure you mark 
your answer in the correct space on 
your answer sheet. 

We are now ready to begin the test. 
Please use the end of your pencil with 
the eraser to break the seal on the test 
booklet, but do not open the booklet. 
The Listening Comprehension will be 
administered by a tape recording. All 
necessary instructions are on the 
recording. If you cannot clearly hear 
the recording, please raise your hand. 
Otherwise I will not stop the 
recording until it has come to the end. 
Any questions? [Wait for response] 

Now I will start the tape. 



Without Structure Section 

DAY ONE 

Administering the listening 
Comprehension Section 

[NOTE: This section is always administered 
FIRST] 

Prepare the examinees to begin the test by 
saying: 

This test consists of two sections: 
Listening Comprehension and Reading 
Comprehension. Today you will take 
the Listening section. The total testing 
time is about 40 minutes. 



There are 50 questions in the 
Listening Comprehension section of 
this test. Your answers should be 
recorded on Side J of the answer 
sheet. Always make sure you mark 
your answer in the correct space on 
your answer sheet. 

We are now ready to begin the test. 
Please use the end of your pencil with 
the eraser to break the seal on the test 
booklet, but do not open the booklet. 
The Listening Comprehension will be 
administered by a tape recording. All 
necessary instructions are on the 
recording. If you cannot clearly hear 
the recording, please raise your hand. 
Otherwise I will not stop the recording 
until it has come to the end. Any 
questions? [Wait for response] 

Now I will start the tape. 
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Begin playing the test tape. The tape will 
start with the words, •This is the Polish 
Proficiency Test.* While these directions are 
being read, walk about the room to make 
sure that the tape is clearly audible to all 
examinees and make any necessary 
adjustments. The total playing time is 
approximately 40 minutes. Please do not 
lea*? the testing room during this time. 
During the playing of the test tape, it would 
be advisable to walk quietly around the room 
to make sure that the playback volume is 
satisfactory and that the examinees are filling 
in their answer sheet properly. 

At the end of the tape, the announcer will 
say: "This is the end of the Listening 
Comprehension section. Please close your 
test booklet and wait for further 
instructions." You should now stop the tape 
and say: 

We will now have a short rest break. 
Please insert your answer sheet in 
your test booklet and close your 
booklet. We will begin the next section 
of the test in three minutes. [You may 
leave this room if you wish. J 



Administering the Structure 
Section 

Begin the Structure section of the test by 

saying: 

There are 35 questions in the 
Structure section of this test. Your 
answers should be recorded on Side 2 
of the answer sheet, beginning with 
question J 01. Always make sure you 
mark your answer in the correct space 
on your answer sheet. 



Begin playing the test tape. The tape will 
start with the words, "This is the Polish 
Proficiency Test." While these directions are 
being read, walk about the room to make 
sure that the tape is clearly audible to all 
examinees and make any necessary 
adjustments. The total playing time is 
approximately 40 minutes. Please do not 
leave the testing room during this time. 
During the playing of the test tape, it would 
be advisable to walk quietly around the room 
to make sure that the playback volume is 
satisfactory and that the examinees are filling 
in their answer sheet properly. 

At the end of the tape, the announcer will 
say: "This is the end of the Listening 
Comprehension section. Please close your 
test booklet and wait for further 
instructions." You should now stop the tape 
and say: 

This is the end of the test for the day. 
Please stop working now. Look over 
your answer sheet carefully. Be sure 
all the marks you made are dark and 
heavy. Erase any marks on your test 
booklet. [Pause] 

Please insert your answer sheet in 
your test booklet and close your 
booklet. Do not leave this room until 
you are told to do so. 



Finishing the Test [Day One] 

Immediately collect the test booklet from 
each examinee, making sure that the answer 
sheet has been inserted. Be sure to account 
for all test booklets distributed. When all test 
booklets and answer sheets have been 
accounted for, and when the examinees are 
reminded of the next testing date, the 
examinees may be dismissed. 
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We will now start the Structure 
section of this test. Please turn to 
page 25 in your test booklet and find 
'Section III - Structure. 9 There is a 
total working time of 30 minutes for 
this section. You may begin working 
on this section as soon as you have 
finished reading the directions. 

[START TIMER IMMEDIATELY] 

After 25 minutes, inform examinees of the 
time remaining [Do NOT stop and reset the 

timer]: 

There are 5 minutes remaining in this 
section of the test. 

After 30 minutes, inform examinees: 

This is the end of the test. Please stop 
working now. Look over your answer 
sheet carefully. Be sure all the marks 
you made are dark and heavy. Erase 
any marks on your test booklet. 
[Pause] 

Please insert your answer sheet in 
your test booklet and close your 
booklet. Do not leave this room until 
you are told to do so. 



Finishing the Test [Day One] 

Immediately collect the test booklet from 
each examinee, making sure that the answer 
sheet has been inserted. Be sure to account 
for all test booklets distributed. When all test 
booklets and answer sheets have been 
accounted for, and when the examinees are 
reminded of the next testing date, the 
examinees may be dismissed. 



DAY TWO 



DAY TWO 



After all examinees have been seated, 
distribute the standard answer sheets and test 
booklets to the students, making sure they 
each receive their own copy, 

Give the following instructions after the 
materials have been distributed: 

Please do not open your test booklet. 
For this test, you are going to give all 
your answers on the answer sheet. Do 
not write anything on the test booklet. 
You must use a No. 2 pencil for 
marking your answers. If you do not 
have one, please raise your hand. 

Distribute pencils as necessary. 



After all examinees have been seated, 
distribute the standard answer sheets and test 
booklets to the students, making sure they 
each receive their own copy. 

Give the following instructions after the 
materials have been distributed: 

Please do not open your test booklet. 
For this test, you are going to give all 
your answers on the answer sheet. Do 
not write anything on the test booklet. 
You must use a So. 2 pencil for 
me^king your answers. If you do not 
have one, please raise your hand. 

Distribute pencils as necessary. 



Administering the Reading Comprehension 
Section 

Begin the Reading Comprehension section of 
the tesi by saying: 

There are 50 questions in the Reading 
Comprehension section of this test. 
Your answers should be recorded on 
Side 2 of the answer sheet, beginning 
with question 57. Always make sure 
you mark your answer in she correct 
space on your answer sheet. 

We will now start the Reading 
Comprehension section. Please turn to 
page 9 in your test booklet and find 
'Section It - Reading 
Comprehension. m There is a total 
working time of 50 minutes for this 
section. You may begin working on 
this section as soon as you have 
finished reading the directions. 

[START TIMER IMMEDIATELY] 



Administering the Reading Comprehension 
Section 

Begin the Reading Comprehension section of 
the test by saying: 

There are 50 questions in the Reading 
Comprehension section of this test. 
Your answers should be recorded on 
Side 2 of the answer sheet, beginning 
with question 57. Always make sure 
you mark your answer in the correct 
space on your answer sheet. 

We will now start the Reading 
Comprehension section. Please turn to 
page 9 in your test booklet and find 
" Sect ion It - Reading 
Comprehension. m There is a total 
m writing time of 50 minutes for this 
section. You may begin working on 
this section as soon as you have 
finished reading the directions. 

[START TIMER IMMEDIATELY] 
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After 40 minutes, inform examinees of the 
time remaining [Do NOT stop and reset the 
timer]: 

There are 10 minutes remaining in 
this section. 

After 45 minutes, inform examinees: 

There are 5 minutes remaining in this 
section. 

After 50 minutes, inform examinees: 

This is the end of this test. Please stop 
working now. Look over your answer 
sheet carefully. Be sure all the marks 
you made are dark and heavy. Erase 
any marks on your test booklet. 
[Pause] 

Please insert your answer sheet in 
your test booklet and close your 
booklet. Do not leave this room until 
you are told to do so. 



Finishing the Test [Day Two] 

Immediately collect the test booklet from 
each examinee, making sure that the answer 
sheet has been inserted. Be sure to account 
for all test booklets distributed. When all test 
booklets and answer sheets have been 
accounted for, the examinees may be 
dismissed. 



After 40 minutes, inform examinees of the 
time remaining [Do NOT stop and reset the 
timer]: 

There are 10 minutes remaining in this 
section. 

After 45 minutes, inform examinees: 

There are 5 minutes remaining in this 
section. 

After 50 minutes, inform examinees: 

This is the end of this test. Please stop 
working now. Look over your answer 
sheet carefully. Be sure all the marks 
you made are dark and heavy. Erase 
any marks on your test booklet. 
[Pause] 

Please insert your answer sheet in 
your test booklet and close your 
booklet. Do not leave this room until 
you are told to do so. 



Finishing the Test [Day Two] 

Immediately collect the test booklet from 
each examinee, making sure that the answer 
sheet has been inserted. Be sure to account 
for all test booklets distributed. When all test 
booklets and answer sheets have been 
accounted for, the examinees may be 
dismissed. 
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AFTER THE TEST 



Checking Collected Materials 

Immediately after you have finished administering the test, please do the following: 

1. Completely rewind the test tape. 

2. Separate the answer sheets from the test booklets. 

3. Scan the answer sheets to make sure that all sections have been filled in properly. 

4. Flip through the test booklets to make sure that there are no pencil marks or missing pages. 
Report any defective or missing test booklets or test tapes on the Supervisor's Report Form. Also 
be sure to include any other irregularities such as cheating (give the examinee's name), 
malfunctioning equipment, mistiming, absence of examinees due to illness, etc. 

5. Compare the test booklet number on each examinee's test booklet against the number on side 
two of the answer sheet to make sure they are the same. 

Returning T est Materials to CAL 

Within three days after the test is given, please return the materials requested on the Test Materials 
Checklist to CAL. Be sure to check off all materials being returned on the Test Materials Checklist 
and fill out and sign Supervisor's Report Form. Your comments and suggestions can be placed on 
the back of the Supervisor's Report Form. These comments, regarding any aspect of the test 
including time allotment, clarity of the instructions and questions, etc., are all highly appreciated. 



THANK YOU FOR YOUR ASSISTANCE! 
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PPT 

(POLISH PROFICIENCY TEST) 



EXAMINEE HANDBOOK 



Preliminary Edition 
Fall, 1992 



This handbook is for your use in preparing for the Polish Proficiency Test (PPT). It 
contains a description of the test, instructions for registering for the test, procedures to 
follow on the day of the test, information about your scores, and sample test questions. 
Please familiarize yourself with the information in this handbook before taking the test and 
retain it for use in interpreting your test score. 



CAL 

Copyright © 1992 by Center for Applied Linguistics, Washington, DC. All rights reserved. 
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GENERAL INFORMATION 



The Polish Proficiency Test (PPT) was developed by the Division of Foreign Language Education 
and Testing at the Center for Applied Linguistics (CAL) in 1992 with the collaboration of Polish 
language scholars and teachers in the United States and Canada. Funding for the test development 
project was provided through a grant from the U.S. Department of Education. 

The PPT is designed to evaluate the level of proficiency in Polish attained by American and other 
English-speaking learners of Polish. The test is designed to be most appropriate for students who have 
completed one or more years of Polish language study at the college level (or its equivalent). 

Scores from the PPT can be used in making screening and placement decisions related to the 
following: 

• admission to a Polish study program 

• placement within a Polish study program 

• exemption from a Polish language requirement 

• application for a scholarship or an appointment 

• competency testing upon exit from a Polish program 

• certification of Polish language proficiency for career placement 

The PPT consists of 135 4-option multiple-choice items. Each examinee receives a test booklet 
containing the test questions and a separate, machine-scoreable answer sheet. All responses to the 
multiple-choice items are indicated on this sheet. 

The PPT is divided into three sections: Listening Comprehension (50 items), Reading 
Comprehension (50 items) and Structure (35 items). In the Listening Comprehension section, you will 
hear listening passages consisting of utterances spoken by a single individual, excerpts from 
conversations between two speakers, and longer single-speaker passages such as news broadcasts, 
advertisements, public interest announcements and so forth. Following each passage in Polish, you will 
be asked a question in English for which you must choose the correct answer from among four options 
printed in English in the test booklet. 

Each item in the Reading Comprehension section will present you with a Polish text, followed by 
a question about the text and four possible responses, printed in English. This section is designed to 
assess your ability to understand a variety of printed texts ranging from a few words (e.g., street or 
store-front signs, announcements) to passages of paragraph length (e.g., excerpts from newspaper or 
magazine articles, personal and official correspondence, popular stories). 

The Structure section has two parts. In the first, you must choose from among four options the 
correct word or phrase that is missing from a text. In the second, you must indicate from among four 
underlined words which one contains an error. 

Questions to all items are printed in a single test booklet. Your answers will be marked on a 
separate machine-scoreable answer sheet. Making any marks on the test booklet may result in 
cancellation of your test score. The total testing time for the PPT is approximately two and a half hours. 
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Sample items from the PPT appear in the Appendix of this manual. Carefully study the directions 
and the sample items. They will give you a good idea of the types of questions you will find on the 
actual PPT. 

REGISTERING FOR THE TEST 

The PPT is administered only through colleges or other institutions that offer a Polish study 
program. There are two ways to register: institutional and individual registration. If your instructor or 
institution has arranged for you to take the test (institutional registration), your institution will inform 
you of the date, time, and location of testing and of any testing fees you may be charged. 

If you wish to register for the test as an individual, you must first contact a local institution and 
register there with a test supervisor, who will administer the test. The date and time you select to take 
the test will be arranged by the supervisor at your mutual convenience. Your test supervisor may be a 
Polish instructor at your school. If you are not currently enrolled in any Polish language program, you 
may contact the Coordinator of the Polish Proficiency Test at CAL (202-429-9292) and request the 
names of institutions in your area at which you may be able to take the test. 

When registering for the test, you will need to bring with you a photo identification card (driver's 
license, student ID, or passport). You will also be asked to give your name and social security number 
to the test supervisor. The current testing fee for the PPT is $20, which you may pay to the supervisor 
by cash or check when you register or when you report to the testing center on the day of the test. If 
you choose to pay by check, confirm with your supervisor whether the check should be made payable 
to the supervisor or to the participating institution. Do not write a check to CAL. Some institutions 
may cover the testing fee on your behalf, in which case you would not need to pay any fee. 

PREPARING FOR THE TEST 

As a test of general language proficiency, the PPT is not based on any particular textbook or course 
of study, but on the kinds of language-use situations that are encountered in real-life contexts outside 
of the academic setting. Because this type of proficiency can be achieved only through continued study 
and practice with the language, it will probably not be helpful for you to study intensively any particular 
textbook just prior to taking the test. The suggested preparation for the test is to carefully study the test 
directions and sample questions in this handbook until you arc thoroughly familiar with the testing 
procedures and question types used in the test you will be taking. 

On the day of the test, you must report to the testing center at least 10 minutes before the test is 
scheduled to begin; no one will be admitted after the test has begun. Remember to bring the following 
items with you: 

1 . a photo ID (driver's license, student ID, or passport); 

2. $20 testing fee (if it is not waived by your institution and you have not prepaid during 
registration); and 

3. several sharpened medium-soft (No. 2) pencils with erasers. 

If you forget to bring the first two items, you will not be admitted to the testing room. There can 
be no exceptions to this rule. 
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The following materials are prohibited in the testing room: books, dictionaries, personal recording 
devices, photographic devices, pens, notes, and papers. Any cheating (giving or receiving assistance, 
taking the test for someone else, copying or recording any test material, etc.) will result in dismissal 
from the testing room and cancellation of your test scores. The testing fee will not be refunded under 
such circumstances. 

REQUESTING SCORE REPORTS 

After the test has been administered, the Center for Applied Linguistics will score the test and send 
a score report roster to the institution. You may then check with the institution for your test results. The 
time between CAl/s receipt of the completed test materials and the sending out of the score roster is 
about 2-3 weeks. 

CAL also provides score reporting services to individual examinees for the purpose of official 
certification of their Polish language proficiency. This official score report can be sent either to you or 
to an institution you designate. You must use the Request Form for Official Score Reports on the last 
page of this handbook. Mail this form with $6.00 for each score report to be sent to the Coordinator, 
Polish Proficiency Test, Center for Applied Linguistics, 1118 22nd Street, N.W., Washington, DC 
20037. Payment may be made with a check drawn on a bank in the U.S. or by money order. Score 
reports will be sent within 2 weeks following receipt of your request. 

INTERPRETING YOUR TEST SCORES 

The Meaning of the Score on the PPT Scale 

Scores on the PPT for Listening, Reading and Structure are reported as scores on the PPT Scale. 
The scaled scores are based on your Polish language proficiency as demonstrated by the difficulty of 
the items you answered correctly, rather than only on the number of items you answered correctly. 

Scores on the PPT Scale can be understood in several ways. First, a score on the PPT Scale can 
be interpreted in a normative manner directly. This means that your scaled score can be compared to 
the scores of a group of 70 examinees typical of North American students studying Polish at the college 
level in any given year. Sixty-seven percent of this group were in their first year of Polish study, 24% 
were in their second, and 9% were in their third (or higher) year. Forty-eight percent were male and 
52% were female. Fifty-eight percent were of Polish ethnicity. Sixty-four percent of the group indicated 
that they spoke no Polish at home, while 3% spoke Polish at home "most of the time," 6% spoke Polish 
at home "less than half the time," and 27% spoke Polish "very infrequently" at home. 

A score of 100 on any section reflects the average (mean) performance of these 70 students on the 
PPT. If you scored 100 on any section, you have demonstrated average proficiency when compared to 
this group. In statistical terminology, each 20 points on the PPT Scale is equal to a standard deviation. 
This means that approximately 95% of all scores from a group of examinees comparable to the norm 
group will fall in the range from 60 to 140 on the PPT Scale. 



Using the Table of Quartile Distributions 



To better compare your scores with those of examinees in your year of study, use Table 1 , which 
provides quartile scores on the PPT Scale for examinees in the first, second, or third (or more) year 
of Polish language study. Quartiles represent the bottom, second, third and top quarter of the score 
distribution. If your score places you in the second quartile, this indicates that you performed in the 
second 25% of all students in your subgroup, and better than the bottom 25% of all the examinees in 
that subgroup. A score in the fourth quartile indicates that you scored in the top 25% of all other 
examinees in the subgroup; i.e., better than 75% of all other examinees. 

To use Table 1, first categorize yourself as either a first, second, or third (or more) year student 
of Polish. Then find the category in the table that contains your Listening, Reading or Structure section 
score. Finally, look over in the left-most column to find the quartile corresponding to your score. 



Table 1 

Quartile Distributions for the PPT 
by PPT Section and Year of Study 







First Year 




Second Year 


Third Year or Higher 


Quartile 


List 


Read 


Su- 


List 


Read 


Str 


List 


Read 


Str 


4th 

(76-100%) 


99 and 
above 


105 and 
above 


it* and 
above 


121 and 
above 


119 and 
above 


123 and 
above 


139 and 
above 


144 and 

above 


154 and 
above 


3rd 

(51-75%) 


92-98 


93-104 


93-105 


105-120 


107-118 


110-122 


130-138 


127-143 


122-153 1 


2nd 
I (26-50%) 


83-91 


8: -92 


84-92 


97-104 


99-106 


104-109 


101-129 


109-126 


108-121 


9 ist 

| (1-25%) 


82 and 
below 


82 and 
below 


83 and 
below 


96 and 

below 


98 and 
below 


103 and 
below 


100 and 
below 


108 and 
below 


107 and 
below 



For example, if you are a student in a first year college-level Polish class and received a PPT 
Scale score of 95 on the Listening Comprehension section, you would first look under the 
column for first year students, then find the column for listening and under this column, find the 
score group that contains your score. Your score of 95 is contained in the 92-98 range. This 
corresponds in the left hand column to the third quartile. Thus, you know you have scored above 
50% of ail first-year examinees in the Listening Comprehension section, but below the top 25% 
of students in first year Polish. Note that your PPT Scale score of 95, when compared with the 
70 examinees in the norming group, indicates you scored below their average score of 100. 

Using the Group Means for Polish, non-Polish Speakers 

A number of Polish lanfiage students use Polish in the home. Such examinees may have an 
advantage on a test of general language proficiency. However, since these examinees formed 
only a small minority of the total number of students in the norming group, and since they do 
comprise a part of the total group of North American students studying Polish, they were 
included in Table 1 . Nonetheless, you may want to understand the meaning of your test scores 
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in terms of whether you speak Polish at home or not. Table 2 presents the average PPT Scale 
score for each section by year of study for examinees who indicated that they do not generally 
speak Polish at home (labeled English) and for those who indicated that Polish is spoken about 
half the time or more (labeled Polish). If no average is shown, there were not enough members 
in the subgroup to calculate a reliable average score. 



Table 2 

Mean Scores on the PPT Scale 
by Level and Home Language 





Listening 


Reading 


Structure 


Year of Study 


English 


Polish 


English 


Polish 


English 


Polish 


First Year 


92 




93 


96 


94 


96 


Second Year 


107 




107 




110 




Third Year (or higher) 


124 




119 


130 


121 


129 



To use Table 2, find the subgroup to which you belong and compare your score to that 
subgroup's mean score. For example, if you were a third-year student who scored 125 on the 
Reading Comprehension section and speak Polish at home, your score would place you slightly 
below the mean of 130 for your peers. On the other hand, if in the same situation you belonged 
to the -English- category, your score of 125 would be somewhat above the average of 1 19. 



APPENDIX 
Practice Questions for the 
Polish Proficiency Test (PPT) 



Directions: 

In this Appendix, you will find the actual test directions followed by practice 
items for each of the types of questions found in the PPT. Carefully work through 
these items as if you were taking the actual test. Answers to these items appear at 
the end of this section. 

Remember: Your score on the test will be based only on questions you answer 
correctly, so it will be to your advantage to answer every question, even though 
you may not be sure that your answer is correct. There is no penalty for guessing. 
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Practice Questions for the PPT 



Listening Comprehension Section 



NOTE: All spoken material for this section is contained on a tape recording which will be 
played to you at the time of testing. For the practice questions below, the material printed in 
italics represents spoken passages that will not appear in print. Inside the small brackets, [W] 
indicates a woman's voice, [M] a man's voice, and [A] the English announcer's voice. 



Directions: For each question in this part, you will hear a short spoken statement or 
question in Polish, followed by a spoken question about it in English. From the four 
choices, labeled A through D, printed in your test booklet, choose the one that correctly 
answers the question and mark the corresponding space on your answer sheet. Listen to the 
following example: 

[W] Mamusiu, Tatusiu, wychodze do idna z koletankami. 
[A] With whom is the speaker going to the movies? 

(A) Parents 

(B) Friends 

(C) Brothers 

(D) Sisters 

Of the four choices, B, "Friends," is the correct answer to the question, "With whom is 
the speaker going to the movies?" If this were one of the test questions, you would mark 
the space labeled B on your answer sheet. 

For each question in this part and throughout the test, mark only one of the four spaces, 
A, B, C, or D. 



1 . [W] Prosimy do stolu. 2. [M] Mamy szalenie duio wypadkdw. 



Part One 



[A] 



Where would this statement most 
likely be heard? 



Ludzie jetdtq nieuwainie. 
Wietozotf z tych wypadkdw f ktdre 
zalatwiamy f me powinna m\e£ 
zupelnie miejsca. 



1. 



(A) 
(B) 
(C) 
(D) 



In a home 
At a ticket counter 
In a furniture store 
In a classroom 



2. 



[A] 



Whai is the most likely profession of 
this speaker? 



(A) 
(B) 
(C) 
(D) 



Scientist 
Journalist 



Police officer 
Teacher 
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Practice Questions for the PPT 



Part Two 



Directions; For each question in this part, you will hear a short conversation in Polish 
between two speakers. The conversation will immediately be followed by a spoken question 
about it in English. From the four choices printed in your test booklet, choose the one that 
correctly answers the question and mark the corresponding space on your answer sheet. 
Listen to the following example: 

[W] Przepraszam bardzo, He kosztuje ta szynka? 
[M] 2000 ziotych kilo. 
[F] Poprosze p6i kilo. 

[A] What does the woman want to know about the item? 

(A) Its price 

(B) Its size 

(C) Its weight 

(D) Its quality 

The correct answer to this sample item is A, "Its price." 



3. [W] Tu sq taJde warunki, ie w iadnym 4. [M] Jak tarn twoja Marysia uczy sie w 
normalnym cywilizowanym kraju takie szkole w tym roku? 

targowisko by nie istniaio. Czy to 

moina zmienid, pana zdaniem? [W] No wiesz, w tym roku poszia jut do 



[M] Tak, moina zmienid, przy 
odrobinie dobrej woli 
wlaScicieli tego targowiska, 
jak i wladz dzielnicy. 



[A] According to the man, who needs to 
cooperate to solve the problems of the 
market? 



nowego liceum, ale okazalo sie, ie 
poziom z matematyki jest o wiele 
niiszy mi my$leli§my. WybrcdiSmy to 
liceum bo bylo blisko, cdz w 
przysziym roku zdecydujemy sie jd 
przeniefc do liceum z rozszerzonym 
programem matematycznym. 



3. (A) The police and the owners of the 



[A] According to the woman, on which 
basis did she initially choose the 
school for her daughter? 



market 



(B) The police and the inhabitants of the 
district 



4. (A) Its location 



(B) Its program 

(C) Its teachers 

(D) Its reputation 



(C) The owners of the market and the 
local authorities 



(D) The inhabitants and the local 
authorities 
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Practice Questions for the PPT 



Part Three 



Directions: In this part, you will listen to a variety of spoken passages in Polish, including 
radio broadcasts, announcements, and other types of material. After each passage, you will 
hear a spoken question about it in English. From the four choices printed in your test 
booklet, choose the one that correctly answers the question and mark the corresponding 
space on your answer sheet. 

There is no example question for this part. 



[W] A w naszym programie jut za chwik 6. [W] Do historii przechodzi istmejoca od 

nowy film angielski. ponad czterdziestu lot iddzka 



[A] 



(A) 
(B) 
(C) 
(D) 



When would this television 
announcement most likely be heard? 



Before a news program 
During a sports event 
Before a feature movie 
After a documentary 



Oridestra Polskiego Radia i Telewiyi. 
Pozostawia po sobie tysioce nagran 
magnetofonowych i ptytowych. 
Orkiestrz wyparty z anteny nagrama 
zagranicznej konkurencji. 



[A] What is the main topic of this news 
item? 



(A) The closing of a radio station 



(B) The anniversary of a record company 



(C) The disbanding of an orchestra 

(D) The reissuing of old tapes and 



records 



Practice Questions for the PPT 



Reading Comprehension Section 
Total Testing Time: 50 minutes 



Directions; This section consists of printed Polish texts ranging in length from a few 
words to short paragraphs. Following each is a question in English. On the basis of the 
information given in the Polish text, choose the correct answer, (A), (B), (C), or (D), to 
each question. Then mark the corresponding letter on your answer sheet next to the number 
of the question. 



There is no example question for this part. Begin work as soon as you have understood 
these directions. You will have 50 minutes to complete this section. 



Premier na Dalekim Wschodzie 

7. The article following this newspaper 
headline most probably discusses 

(A) an election 

(B) an official visit 

(C) a meeting with advisors 

(D) a political faux pas 



Polska liczy obecnie ponad 32 miliony 
mieszkaric6w i pod wzgl^dem liczebno£ci 
zajmuje si6dme miejsce w Europie i 
dziewi§tnaste w Swiecie. 

8. According to this passage (written in 
1970), which position in the world does 
Poland occupy in terms of population? 

(A) Sixth 

(B) Seventh 

(C) Twelfth 

(D) Nineteenth 



Pierwszym 50 Czytelnikom, 
ktfrzy juz w najblizszy pi*tek, 
6 wrzetaia, w godz. 15-16. 
odwiedx* nssz* redakcjc, funduje- 
my - z okazji jubileuszu 45* 
-lecit .Expressu Wieczornego" 
— bilety na sobotnio-niedzielne 
dyskoteki w studeockiin klubie 
.STODOLA" przy ul. Batorego 

9. According to this contest announcement, 
what must readers do to win the prize? 

(A) Participate in a dance 

(B) Write an essay 

(C) Visit an office 

(D) Purchase special tickets 



Caikowicie lub cz^Sciowo 
niewykorzystane bilety powinny by£ 
przedstawione w celu uzyskania zwrotu 
oplaty w okresie 30 dni od daty 
wygaSniqcia ich wazno^ci. Zwrotu za 
bilet dokonuje to biuro, w kt6rym zostat 
on wystawiony i optacony. 

10. What procedure does this information 
found on an airline ticket explain? 

(A) How to exchange the ticket 

(B) How to reconfirm reservations 

(C) How to get a refund 

(D) How to label luggage 
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Practice Questions for the PPT 



Structure Section 

Total Testing Time: 30 minutes 



General Directions: The Structure section is composed of two parts. The total testing time 
for the section is 30 minutes. Go directly to Part Two after you finish Part One. 

Part One 

Directions: In this part there are five Polish passages, each with five missing portions. The 
missing portions are indicated by an underlined item number. Beneath each passage are five 
items labeled with item numbers corresponding to those in the passage. For each item, 
choose one of the four options, (A), (B), (C), or (D), which completes the missing portion 
of the passage in a meaningful and grammatically correct way. Then mark the 
corresponding letter on your answer sheet next to the number of the missing portion. 



Study the following example: 

29 bm. przypada 50 rocznica Smierci Ignacego Paderewskiego. Tego wlainie dnia 
Teatr Wielki Exl z premiers jedynej jego opery, „Manru\ Libretto Alfreda 
Nossiga oparte jest na Ex2 popularnej powie&i Jdzefa Ignacego Kraszewskiego 
„Chata za wsia*. 

Jest to ostatnia premiera Teatru Wielkiego w tym sezonie, a juz w lipcu i sierpniu- 
-przerwa urlopowa. 

Exl. (A) wystepowad Ex2. (A) motywy 

(B) wystepuja (B) motywach 

(C) wystspuje (C) motywami 

(D) wystepuje (D) motywdw 

For Exl, the correct answer is (D) and for Ex2, the correct answer is (B). These are the 
only choices that complete the missing portions of the passage in a meaningful and 
grammatically correct way. Begin work on this part as soon as you have understood the 
directions. 



Practice Questions for the PPT 



W ciagu 11 lat ioinierze zawodowi i rezerwi&i zobowiazani sa 12 
ponownej przysiegi wojskowej, ktdra bedzie sic korfczyla 13 „Tak mi dopomdz 
B6g\ Zolnierz skladajacy przysiege moze 14 od wypowiedzenia ostatniego zdania 
roty - zadecydowal 15 czwartkowego posiedzenia Parlament. 



(A) 


pie<5 


13. (A) 


s!6w 


15. (A) 


na czas 


(B) 


piatka 


(B) 


slowem 


(B) 


do 


(C) 


piociu 


(C) 


slowach 


(C) 


podczas 


(D) 


piatka 


(D) 


stowami 


(D) 


od 


(A) 


ziozeniem 


14. (A) 


przestapil 






(B) 


aby zlozyd 


(B) 


odstapil 






(C) 


zlozony 


(C) 


przystapil 






(D) 


do zlozenia 


(D) 


wstapid 







Part Two 

Directions: In this part, there are three Polish passages, each with five numbered lines. 
In each numbered line, there are four underlined words. These are marked (A), (B), (C), 
and (D). Find the ONE underlined word that must be changed in order for the line to be 
grammatically correct. Then find the number of the line on your answer sheet and mark 
the letter that corresponds to your answer. 

Study the following example: 

Od pierwszego lipca importerzy piwa i wina musza mied specjalne 

Ex3. pozwolenia na przywdz . Pozwolenie taki moga dostad tylko osoby, 

(A) (B) 1q "(D)" 

Ex4. ktdre ma licencje ministra rynku wewntfrznego na obrdt 

(A) (B) (Q "(D)" 

hurtowy napojami alkoholowymi. Ograniczenie jest tymczasowe i 

obowiazuje do 31 grudnia biezacego roku. 

In Ex3, choice (B) should read "takie" in order to make the line grammatically correct. 
Therefore you should mark answer (B) on your answer sheet. In Ex4, choice (A) should 
read "maja" for the line to be grammatically correct. You should mark answer (A) on your 
answer sheet. Begin this part of the test as soon as you have understood the directions. 
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Practice Questions for the PPT 



A czy z czasem pani obrazy zmieniaja sic? 

- 16. Niewiele, dawniej byly podobni , moze jedynie trochg spokojniejsze, 
(A) (B) (C) (D) 

17. moze bardziej narracyjne , na pewnym bardziej slodkie. Ale trudno mi 

(A) (B) (C) (D) 

18. o tym opowiadam. Ten sam problem mialam na obronie dyplomu , 

(A) (B) (C) (D) 

19. Komisja pytala mnie, co mam dla powiedzenia. 

(A) (B) (Q (D) 

20. Powiedziatam im, ze nic, widad natomiast, co mam do pokazywania. 

(A) "(B) (C) (D) 



Answers to Practice Questions 



V 03 


D SI 


D "01 




D '61 


e -fi 


0 6 


v -t 


V 81 


a ei 


a 8 


d e 


9 LI 


a z\ 


8 'L 


D 'Z 


8 91 


D 'II 


0 9 


V I 
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Polish Proficiency Test 
REQUEST FORM FOR OFFIC IAL SCORE REPORTS 



Print the required information below and enclose a check or money order for $6.00 made payable to 
PPT/Center for Applied Linguistics for each score report you request (i.e., $6.00 if requesting one 
score report, $12.00 if requesting two). Detach and mail this form together with the correct payment 
to: Coordinator, Polish Proficiency Test, Center for Applied Linguistics, 1118 22nd Street, N.W., 
Washington, DC 20037. 



Examinee Name: 



Last 



First 



MI 



Social Security Number: 
Test date: / 



(month / year) 



Testing Center: 



Test Supervisor (if known): 



Score Report 1 



Score Report 2 



Name of Score Recipient 



Name of Score Recipient 



Institution (Testing Center) 



Institution (Testing Center) 



Department 



Department 



Number and Street 



Number and Street 



City 



City 



State/Zip Code 



State/Zip Code 



Your signature 



Your signature 



Date 



Date 



Print the name and address of the institution to which you want a PPT official score report sent. If 
you want the report sent to you, give your own name and address. Photocopy this form if more than 
two score reports are requested. 
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The Center for Applied Linguistics was chartered in 1959 as a non-profit 
corporation in Washington, DC with initial support provided by the Ford 
Foundation. While historically CAL has developed language tests to meet the needs 
of its various domestic and international projects, in 1986 CAL's language testing 
initiatives were centralized in the Foreign Language Education and Testing 
Division. The Division produces language tests for use in governmental, 
educational, and occupational settings. For information on CAL's test development 
capabilities, contact Dr. Charles W. Stansfield, Director, Division of Foreign 
Language Education and Testing, CAL, 1118 22nd Street, NW, Washington, DC 
20037, (202) 429-9292. 
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PREFACE 



The PPT Test Interpretation Manual has been prepared for individuals who are interested in the 
Polish Proficiency Test (PPT) offered by the Center for Applied Linguistics (CAL): for 
individuals considering using the test, for those who administer the test, and for those who use 
or interpret scores attained on the test. It presents an overview of the testing program, the uses 
and purposes of the tests, information on the interpretation of test scores, and the statistical 
characteristics of the test. 



The Polish Proficiency Test was developed in the Division of Foreign Language Education and 
Testing at CAL through support provided by the International Research and Studies Program of 
the U.S. Department of Education. The test is the result of collaboration of the CAL with 
Polish language teaching professionals throughout North America. Below is a list of primary 
individuals who worked on developing the test. 



Test Development Team 



Charles W. Stansfield 
Dorry Mann Kenyon 
Xixiang Jiang 
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GENERAL INFORMATION 



History of the PPT 

The Polish Proficiency Test (PPT) was developed by the Division of Foreign Language 
Education and Testing at the Center for Applied Linguistics (CAL) in 1992 with the 
collaboration of Polish language scholars and teachers in the United States and Canada. Funding 
for the test development project was provided through a grant from the U.S. Department of 
Education. 

Test Content 

The PPT is designed to measure the general proficiency in listening and reading attained by 
American and other English-speaking learners of Polish. In keeping with its intended purpose 
as a measure of general language proficiency, the major focus of the test is on assessing the 
examinee's ability to function in situations typical of real-life language-use situations. It was 
designed for examinees at proficiency levels from Novice High to Superior according to the 
ACTFL Proficiency Guidelines (American Council on the Teaching of Foreign Languages, 
1986). In terms of length of Polish study, the PPT has been designed to be appropriate for 
students who have completed one or more years of college level Polish language instruction. The 
test is suitable for students at the end of their first year of study at the college level. However, 
students who have completed less than one year of college-level study (or its equivalent) will 
most likely find the test too challenging. 

Test Format 

The PPT consists of 135 4-option multiple-choice items. Each examinee receives a test 
booklet containing the test questions and a separate, machine-scoreable answer sheet. All 
responses to the multiple-choice items are indicated on this sheet. 

The PPT is divided into three sections: Listening Comprehension (50 items), Reading 
Comprehension (50 items) and Structure (35 items). In the Listening Comprehension section, 
examinees hear listening passages consisting of utterances spoken by a single individual, excerpts 
from conversations between two speakers, and longer single-speaker passages such as news 
broadcasts, advertisements, public interest announcements and so forth. Following each passage 
in Polish, examinees are asked a question in English for which they must choose the correct 
answer from among four options printed in English in the test booklet. All listening passages 
have been excerpted from authentic sources. To ensure the quality of the recording, the passages 
have been rerecorded. All Polish voices on the cassette are native Polish speakers, carefully 
selected for voice quality and authenticity of expression. The spoken material is delivered with 
natural pacing and intonation. 

Each item in the Reading Comprehension section presents examinees with a Polish text, 
followed by a question about the text and four possible responses, printed in English. This 
section is designed to assess the examinees' ability to understand a variety of printed texts 
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ranging from a few words (e.g., street or store-front signs, announcements) to passages of 
paragraph length (e.g., excerpts from newspaper or magazine articles, personal and official 
correspondence, popular stories). All test questions deal with either the factual content of the 
passage or with straightforward inferences based on the passage. There are no texts or questions 
requiring a background in Polish literature. 

The Structure section has two parts. In the first, examinees must choose from among four 
options the correct word or phrase that is missing from a text. In the second, examinees must 
indicate from among four underlined words which one contains an error. 

For more information on the content of the PPT, see the section on Content Validity on page 
10 of this manual. The PPT Examinee Handbook, available from CAL, contains examples of the 
types of items found on the PPT. 

Testing Time 

The PPT is designed so that each section may be separately administered. The testing time 
for the Listening Comprehension section is 40 minutes, with 50 minutes allotted for the Reading 
Comprehension section and 30 minutes for the Structure section. Including time for instructions 
and breaks, the entire test may be administered in a single session lasting approximately two and 
a half hours, or in two sessions of approximately one hour and fifteen minutes each. Programs 
placing a high emphasis on developing proficiency and/or decreased importance on grammar 
may decide to omit the Structure section of the PPT. If it is omitted, the required test 
administration time is decreased by about 30 minutes. 

Complete information on administering the PPT is contained in the PPT Supervisor's 
Manual, which is sent with test orders. If desired, it may be separately requested from CAL. 

USES OF THE PPT 

The primary focus of the PPT is on assessing the examinee's ability to understand spoken 
and written Polish encountered in contemporary, real-life language-use contexts. As such, a 
major use of the PPT is to provide an objective measure of an individual's general proficiency 
in Polish. Another major use of the PPT is to provide Polish-teaching institutions with an 
external measure of the overall effectiveness of their instructional programs. A number of other 
uses are possible as well. The PPT may be used for the following purposes: 

• admission to a Polish study program 

• placement within a Polish study program 

• exemption from a Polish language requirement 

• application for a scholarship or an appointment 

• competency testing upon exit from a Polish program 

• certification of Polish language proficiency for career placement 

• evaluation of a Polish instructional program 
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CAL encourages use of the test by an institution if such use will assist it in making valid 
decisions in one or more of the above applications. However, the institution itself must 
determine whether the PPT is an appropriate test with respect to its intended uses and must 
establish its own level of acceptable performance on the test. 

PPT score users are invited to consult with CAL testing staff about current or intended uses 
of test scores. Furthermore, the testing office can assist institutions contemplating use of the test 
by providing information about the applicability of the test to particular situations and by 
providing assistance in establishing performance criteria on a local basis. If it is necessary to 
contact CAL, correspondence should be directed to: 

Coordinator, Polish Proficiency Test 
Center for Applied Linguistics 
1118 22nd Street, NW 
Washington, DC 20037 
TEL: (202) 429-9292 
FAX: (202) 659-5641 



INITIATING TEST ADMINISTRATION 

Registering for the Test 

The PPT is available to colleges, universities and other institutions in the United States and 
abroad for administration to their students on tests dates of their choice. Administration may l>e 
initiated in two ways. An institution may decide to administer the test to its students. In this 
case, the appropriate person makes arrangements with CAL directly. Alternatively, an individual 
who needs to be tested may initiate administration. In this case, the individual must first contact 
CAL for the name of the nearest PPT testing center. If a test center is not available nearby, the 
individual should contact a local institution that offers a Polish language program. That 
institution may then apply to CAL to administer the PPT to the individual by designating a 
supervisor and filling out an order form (see Ordering the Test below). 

When registering for the PPT, an examinee is required to present a photo identification 
document (e.g. driver's license, student I.D. or passport) to the supervisor. The supervisor must 
verify the identification and record the name and social security number. The $20 testing fee is 
payable either at the time of registration or on the test date. No examinee should be allowed to 
take the PPT without paying the fee in advance. The testing fee can be paid in cash or by a 
check payable to the test center supervisor or to the institution. An invoice for the total testing 
fee will be issued to the institution after the test has been administered and test materials 
returned to CAL. 



Ordering the Test 



To place an order for the PPT, the supervisor must provide CAL with the following 
information in writing: 

1 . title/position and affiliation of the supervisor; 

2. approximate test date; 

3. number of examinees expected; 

4. whether rush delivery is desired (at the institution's expense). 

A test order should arrive at CAL approximately 2-3 weeks before the intended test date to 
allow timely shipment of the necessary test materials. To expedite processing of test orders, it 
is highly recommended that printed PPT Order Forms be used. A copy is printed on the last 
page of this manual. Telephone orders are accepted only under urgent circumstances and only 
from institutions which have administered the PPT previously. 

Upon receipt of the Order Form, CAL will forward to the designated supervisor the 
requested number of PPT test booklets and associated materials, including the test tape for the 
Listening Comprehension section, examinee answer sheets, PFT Examinee Handbooks, the PPT 
Supervisor's Manual and other materials. The PPT Supervisor's Manual contains all instructions 
for the actual test administration. 

It is the responsibility of the supervisor to safeguard and maintain the security of the test 
materials from the time they are received at the institution until they are returned to CAL. It is 
important that the test be administered in strict accordance with the procedures outlined in the 
supervisor's manual. All materials requested on the Test Materials Checklist must be accounted 
for and returned to CAL (see the PPT Supervisor's Manual for detailed instructions). 

Score Reporting and Billing 

Approximately 2-3 weeks following receipt of the returned test materials, CAL will forward 
a roster of scores to the institution. An invoice for the total testing fee, including any delivery 
expense for rush orders, will follow shortly afterward. This invoice must be paid by check or 
money order, payable to: Polish Proficiency Test /Center for Applied Linguistics. 



INTERPRETATION OF TEST SCORES 



The Meaning of Scores on the PPT Scale 

Rather than receiving a raw score on each section of the test (i.e., tlie number of right 
answers), examinees taking the PPT receive scores reported on the PPT Scale. A score on the 
PPT Scale more accurately reflects an examinee's proficiency than the number of right answers 
on each section of the test. In addition, the scaled score contains interpretative information, 
which is explained below. The PPT Scale was developed using Item Response Theory (IRT),' 
using the Rasch model. Scores on the PPT Scale are based on the proficiency demonstrated by 
the examinee in terms of the difficulty of the items the examinee answers correctly. No penalty 
is imposed for guessing, or for incorrect or blank responses. 

Scores on the PPT Scale are interpreted in a normative manner. A score of 100 on any 
section reflects the average (mean) performance of the PPT norming group. Each standard 
deviation in this norming group is equal to 20 points on the PPT Scale. An examinee with a 
score of 100 on any section is of average proficiency based on the norming sample described 
below. An examinee with a score of 140 is two standard deviations above the mean, while an 
examinee with a score of 80 is one standard deviation below the mean. 

Description of the Norming Group 

The PPT norming group is a sample of 70 examinees typical of North American students 
studying Polish at the college level in any given year. At the time of norming, 67% percent of 
the group were in their first year of Polish study, 24% were in their second, and 9% were in 
their third (or higher) year. Forty-eight percent were male and 52% were female. Fifty-eight 
percent were of Polish ethnicity. Sixty-four percent of the group indicated that they spoke no 
Polish at home, while 3% spoke Polish at home "most of the time," 6% spoke Polish at home 
"less than half the time," and 27% spoke Polish "very infrequently" at home. 

Quartile Distributions 

Because of the small number of Polish examinees who could be involved in the norming of 
the PPT, it is impossible to give accurate norms at every score level. Table 1 provides quartile 
scores on the PPT Scale for examinees in the first, second, or third or more year of Polish 
language study. The figures are based on the results of all 129 examinees wto participated in 
the field testing of the PPT. However, not every examinee took every section of the PPT. The 
characteristics of the entire group of 129 examinees did not deviate substantially from those of 
the 70 examinees who comprised the norm group, described above. The norm group is a subset 
of the entire group of 129 examinees. 

Quartiles represent the bottom, second, third and top quarter of the score distribution. A 
score in the second quartile indicates that the examinee performed in the second quarter of all 
students in his or her subgroup, and better than the bottom 25 percent of all the examinees in 
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that subgroup. A score in the fourth quartile indicates that the examinee scored in the top 25% 
of all other examinees in the subgroup; i.e., better than 75% of all other examinees. 



Table 1 

Quartile Distributions for the PPT 
by PPT Section and Year of Study 





First Year 


Second Year 


Third Year or Higher ] 




List 


Read 


Str 


List 


Read 


Str 


List 


Read 


Sir 


4th 

(76-100 %) 


99 and 
above 


105 and 
above 


106 and 
above 


121 and 
above 


119 and 

above 


123 and 
above 


139 and 
above 


144 and 

above 


154 and 
above 


3rd 

(51-75%) 


92-98 


93-104 


93-105 


105-120 


107-118 


110-122 


130-138 


127-143 


122-153 


2nd 

(26-50%) 


83-91 


83-92 


84-92 


97-104 


99-106 


104-109 


101-129 


109-126 


108-121 


1st 

(1-25%) 


82 and 

below 


82 and 

below 


83 and 

below 


96 and 
below 


98 and 

below 


103 and 
below 


100 and 
below 


108 and 
below 


107 and 
below 


1 Examined 


(56) 


(60) 


(69) 


(18) 


(17) 


(19) 


(6) 


(15) 


(15) | 



To use Table 1 in interpreting an examinee's score, first identify the examinee as either a 
first, second, or third (or higher) year student. Then, for each section, find the score group in 
the table that contains the examinee's score. Then read the quartile group from the left hand 
column that corresponds to the examinee's score. 



Example 1 

Susan is a student in a first year college-level Polish class who received a PPT Scale 
score of 95 on the Listening Comprehension section. Under the column for first year 
students, we find the column for listening, and under this column, we find the score 
group that contains her score (92-98). This corresponds in the left hand column to the 
third quartile. Thus, we know she has scored above 50% of all first-year examinees in 
the Listening Comprehension section, but below the top 25% of students in first year 
Polish. 

Example 2 

Michael is a student in a third year college-level Polish class who received a PPT Scale 
score of 125 on the Reading Comprehension section. Under the column for third year 
students, we find the column for reading, and under this column, we find the score group 
that contains his score (109-126). This corresponds in the left hand column to the second 
quartile. Michael has scored above 25% of all second-year examinees in the Reading 
Comprehension section, but below the top 50% of third-year examinees. Note, however, 
that if Michael had been in the second-year category, his Reading Comprehension section 
score of 125 would have placed him in the fourth quartile, indicating that he scored 



above 75% of the second year students, and in the top 25% of all second-year examinees 
in reading. 

Using the Group Means for Polish, non-Polish Speakers 

Some examinees in the norm group indicated that Polish was used in the home more than 
half the time. Such examines may have an advantage on a test of general language proficiency. 
However, since these examinees formed only a small minority of the total, and since they do 
comprise a part of the total group of North American students studying Polish, they were not 
excluded from Table 1. Nonetheless, PPT users may want to understand the meaning of an 
examinee's test scores in terms of whether the examinee indicates that much Polish is spoken 
at home or not. Table 2 presents the average PPT Scale score for each section by year of study 
for examinees who indicated that they do not generally speak Polish at home (labeled English) 
and those who indicated that Polish is spoken about half the time or more Gabeled Polish). 
Underneath each average score is the number of examinees upon which the mean is based. The 
more examinees, the more reliable the mean score. Means for subgroups with less than 3 
members are not shown. 

Table 2 

Mean Scores on the PPT Scale 
by Level and Home Language 
(Number of Examinees in Parentheses) 





Listening 


Reading 


Structure 1 


Year of Study 


English 


Polish 


English 


Polish 


English 


Polish 


First Year 


92 




93 


96 


94 


96 




(53) 


(-) 


(55) 


(5) 


(64) 


(5) 


Second Year 


107 




107 




110 






(15) 


(-) 


(15) 


(-) 


(16) 


(-) 


Third Year 


124 




119 


130 


121 


129 


(or higher) 


(5) 


(-) 


(7) 


(8) 


(7) 


(8) 



To use Table 2, find the group to which the examinee belongs and compare his or her score 
to that group's mean score. 



Example 3 

Susan, the first-year Polish student who received a score of 95 on the Listening 
Comprehension section of the PPT, speaks no Polish at home. Her score is slightly above 
the average score for her peers. 
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Example 4 

Michael, the third-year student who scored 125 on the Reading Comprehension section, 
indicated that he generally speaks Polish at home. His score places him slightly below 
the mean of 130 for his peers. On the other hand, if he belonged to the "English" 
category, his score of 125 would have been slightly above average. 

Performance of Polish Speakers on the PPT 

The PPT is intended for English-speaking learners of Polish. Since the questions and options 
for the Listening and Reading sections as well as the directions for the entire test are in English, 
knowledge of English is required to take the test. Thus, it is impossible to use these tests to 
examine the performance of native Polish speakers (or speakers of a language other than 
English) who do not have a good knowledge of English. Test results may not be accurate for 
any examinee who does not have a good command of English; i.e., a level generally equivalent 
to that of a first-year college student enrolled in the regular program of an American university. 

In the field testing of the PPT, 17 of the 129 examinees indicated that they were of Polish 
ethnicity and spoke Polish at home "most of the time." The responses of these examinees were 
carefully examined to see to what extent, if any, these examinees were getting easier items 
incorrect; i.e., items that they may have missed because of their restricted ability in English 
rather than their proficiency in Polish. None of the responses of the Polish speakers showed any 
incongruities on any of the three sections of the test. On the basis of experience to date, it is 
reasonable to assume that the level of English used in the test will not pose a problem to the 
native Polish speaking examinee who has enough proficiency in English to be enrolled at an 
English-speaking university in North America. 

Availability of Institutional Interpretation of Test Scores 

In addition to using the normative data provided in this manual, institutions may wish to 
carry out further analyses of the scores on a local basis. CAL staff would be pleased to discuss 
such analyses with interested institutions and to facilitate these activities as far as possible. 

STATISTICAL CHARACTERISTICS OF THE TEST 

Reliability 

The reliability of a test is the extent to which it yields consistent results. Thus, high test 
reliability is desirable. 

Table 3 gives the Rasch equivalent of the KR-20 reliability of each section of the final form 
of the PPT based on the data from examinees involved from the field testing. The number of 
examinees for each section is: Listening (96), Reading (110), and Structure (123). 
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Table 3 

Reliability of the 
PPT by Section 



Section 

Listening Comprehension 
Reading Comprehension 
Structure 



Reliability 



.92 
.90 
.88 



The figures presented in Table 3 indicate that the PPT is a highly reliable tests which can 
be used with confidence by programs needing trustworthy measures of Polish language 
proficiency. 

Precision of Measurement 

Any measurement involves a degree of error. The higher the precision of the measurement, 
the smaller the error. In Classical Test Theory, the degree of error is usually indicated by the 
standard error of measurement (SEM). One of the limitations of this approach is that it 
represents the SEM as being the same for all examinees. Test scores, however, are unequally 
precise for examinees at different levels of ability. Item Response Theory (IRT), used in scoring 
the PPT, also provides a measure of the precision of each score on the PPT Scale. In IRT, this 
measure' is called the standard error of the ability estimate. Appendix A contains the standard 
errors for PPT scores, rounded to the nearest whole number. 

A traditional SEM may also be calculated for the PPT on the PPT Scale. These are presented 
in Table 4. Figures have been rounded to the nearest whole number. 



Either the SEM from Table 4 or the standard error of the estimate from Appendix A give 
an indication of how accurate a single test score is. Either can be used to construct confidence 
intervals around PPT scaled scores. The confidence interval takes measurement error into 
account. One way to view this confidence interval is to say that the examinee's true score (i.e., 
the average score an examinee would receive if it were possible to give him or her the test a 
very large number of times) would lie, with 67% certainty, within plus one or minus one 



Table 4 

Standard Error of Measurement (SEM) 
for the PPT 
(in Rounded Scale Score Units) 



Section 
Listening 

Reading 
Structure 



SEM 

6 
7 
7 
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standard error of the given score. Another way to view this is to say that on a re-test, if there 
has been no change in the examinee's true proficiency, the examinee will score within plus or 
minus one standard error of his or her score about 67% of the time. 

Example 5 

Susan received a score of 95 on the PPT Listening section. Using Table 4, we can say 
there is a 67% probability that her true score lies between 89 and 101 (95 minus 6 and 
95 plus 6). Using the more precise estimate from Appendix A, we would say there is a 
67% chance that her true score is between 90 and 100 (95 minus 5 and 95 plus 5). 

A careful examination of Appendix A reveals that the most precise area of measurement on 
the PPT lies between 66 and 136. For scores above 136, the measurement precision of the PPT 
rapidly diminishes. A score of 136 indicates a performance well above the cutoff point for the 
fourth quartile for first and second year students, and in the third quartile for students in the 
category third year and above. In other words, the measurement precision of the PPT should not 
be of concern in interpreting the scores of first and second year Polish students on any section 
of the test. However, test score users should be aware that the PPT may not be able to 
discriminate accurately between very high scoring examinees on the PPT; i.e., examinees who 
score above 135 on any section. Table 2 indicates that such examinees would generally be 
individuals who have completed three or more years of Polish study and who speak Polish at 
home. 

Validity 

Validity refers to the extent to which a test actually measures what it purports to measure. 
The PPT claims to measure an examinee's proficiency in understanding authentic spoken and 
written Polish, and ability to deal with Polish structure. 

The validity of any test cannot be "proven;" it can only be established by the collection of 
evidence that the test is indeed measuring what it purports to measure. Some commonly accepted 
types of evidence include evidence for content, concurrent, and construct validity. Each type of 
validity, and evidence which supports it in the case of the and the PPT, are explained below. 

Content Validity 

Content validity is based on test content and refers to the extent to which the items in a test 
adequately sample the linguistic and behavioral domains of the construct that the test is meant 
to measure. The PPT is intended to be a measure of proficiency in dealing with every-day "real- 
life" Polish. Validity based on content thus entails an examination of the degree to which the 
tests, in their stimulus passages, sample from the corresponding language-use situations the 
examinee might be expected to encounter in real life. 



10 
102 



Content Domains 



The ACTFL Guidelines for listening and reading postulate certain content and functional 
domains as appropriate to the proficiency of an English-speaking learner of Polish at different 
levels of proficiency. The developers of the PPT used authentic listening and reading passages 
and wrote corresponding items to reflect these domains. Content and functional domains can be 
grouped into three categories, which represent increasing levels of proficiency in Polish as a 
second language. These levels may be designated as: 

Intermediate Level-Survival. At this level, the learner can "survive" by performing simple 
daily tasks such as shopping for food, clothes, and incidental necessities. PPT listening 
comprehension items at this level require the understanding of oral language typically heard 
in locations such as a grocery store, train station, bank, bus, and airport. PPT reading 
comprehension items at this level include certain signs, advertisements, storefront 
designations, simple messages wi h a social purpose, and short straightforward instructions 
encountered in daily life. 

Advanced Level-Tourist Resident in Polish Society. At this level, the learner can 
understand social conversation on everyday topics and can read short, descriptive narrations. 
PPT listening comprehension items at this level require comprehension of short conversations 
between two people dealing with general, every day topics, or the comprehension of passages 
such as news items and reports dealing primarily with factual information. PPT reading 
comprehension items at this level require the examinee to comprehend main ideas and some 
facts from passages excerpted from newspaper articles, public notices, personal 
correspondence, and popular stories. 

Superior Level-Foreign Professional in Polish Society. At this level, the learner can 
comprehend oral and written language on virtually any matter of general interest to educated 
native speakers in authentic Polish situations. PPT listening stimuli at this level involve 
conversations between two speakers dealing with topics of a general academic nature such 
as recent advances in health care and current events, or longer discourse excerpted from 
speeches or reports. Questions may deal with the speaker's opinion or attitude, or they may 
require inference on the part of the examinee. Reading stimuli at this level are generally 
longer and more technical and are followed by a questions dealing with basic factual 
information covered in the passage, or inference or analogy based on information presented 
in the passage. The passages cover a wide variety of daily topics such as those that may be 
found in a newspaper or magazine, e.g., results of a public opinion survey, job descriptions 
and vacation travel. ' 

For the Structure section, the PPT tests knowledge of correct Polish syntax. Unlike listening 
or reading comprehension, the Structure section does not involve real-life language-use tasks per 
se, though clearly knowledge of Polish syntax is a part of comprehending spoken and written 
Polish. Items testing recognition of Polish syntax on the PPT are of two types. The first type 
tests knowledge of correct structure by requiring the examinee to choose from among four 
options the correct word or phrase that should be inserted in a blank within a printed paragraph. 



11 



103 



The second type tests structural knowledge by requiring the examinee to find from among four 
underlined words the one that is grammatically incorrect. In this section of the PPT, only Polish 
is used for both the stimulus and response options. 

Appendix B contains a list of the comprehension tasks and topic areas covered in the PPT 
listening and reading comprehension sections. It also contains a list of speaking functions 
covered on the listening section of the test. Appendix C gives information on the grammar topics 
covered in the structure section of the PPT. 

Authentic Sources 

Stimulus passages for listening, reading and structure on the PPT were drawn from authentic 
language sources. Item developers collected passages for the reading comprehension section from 
sources such as Polish language newspapers, magazines, journals, books, street signs, schedules, 
and tickets. For stimulus passages for listening comprehension, they listened to and transcribed 
listening passages from sources such as Polish language news broadcasts, movies, and 
announcements. These texts were modified only to the extent that they needed to be clarified 
when taken out of the larger context. Questions were designed to check comprehension of the 
meaning of passages, and, for more difficult passages on the PPT, to check comprehension of 
opinions, attitudes or inferences contained in the passages. A representative list of sources for 
the listening and reading passages used on the PPT is contained in Appendix D. 

In terms of content validity, then, it can be demonstrated that the PPT items have been 
drawn from real-life use of Polish language. 

Concurrent Validity 

A second type of evidence of validity is concurrent validity. Concurrent validity refers to the 
extent to which a test score correlates with results that may be obtained through the use of 
independent criteria external to the test, measured at the same point in time, to see if expected 
relationships exist. 

For the PPT T the reported level of Polish study may be use as an external criteria may be 
used. It would be expected that scores on the tests would increase as amount of study increases. 
Both Tables 1 and 2 indicate that this is the case for the PPT. In each skill, the median (Table 
1) and average (Table 2) score increases as the level of Polish instruction increases. These tables 
thus provide evidence of the concurrent validity of the PPT as a measure of Polish language 
proficiency. 

Construct Validity 

A third way of examining validity is construct validity. The goal of construct validity is to 
determine whether or not a test measures a single underlying trait. One assumption of the IRT 
model used to develop and score the PPT is that the items are "unidimensionaT; that is, only 
one examinee ability or trait is necessary to account for performance on the test. The fit statistics 
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provided by the IRT analysis provide evidence of the extent to which unidimensionality exists. 
Although perfect fit is impossible, it may be argued that if the overwhelming majority of items 
are appropriately fitting, then there is strong evidence for the construct validity of the test. For 
each :!tm, a standardized outfit statistic is calculated. One common criterion is to consider any 
item for which this statistic is above 2.00 or below -2.00 as misfitting. Using this criterion, only 
3 of the 50 listening items (6%) and 2 of the 50 reading items (4%) on the final form were 
misfitting. This high degree of fit provides strong evidence for the construct validity of these 
measures. In the structure section, 5 of the 35 items (14%) were misfitting. Although 14% 
represents a higher proportion of misfitting items than for the listening and reading sections, the 
figure is low enough to lend some suppon the construct validity for the structure section. It may 
be noted that one of these five items has been completely revised in the final form. 

Validity Summary 

The above discussion provides evidence for the validity of the PPT. CAL plans to continue 
research pertinent to the validity of the test and will update this section of the manual as 
appropriate. 

Intercorrelations Among Test Subscores 

The three sections of the PPT are designed to measure different skills within the general 
domain of Polish proficiency. It is expected that these skills are interrelated; i.e., persons who 
are highly proficient in one skill area will tend to be proficient in the other skills as well. 
However, the intercorrelations are not expected to be perfect. If they were, there would be no 
need to report scores for each section; the subscores would represent the same rather than 
different aspects of language proficiency. 

Table 5 shows the Pearson product-moment correlation coefficients measuring the extent of 
relationships among the three subsections of the PPT based on scores on the final form of the 
PPT for the 86 examinees who took all three sections in the field testing. 



Table 5 

Intercorrelations Among Subscores 

Lis! Read Sir 
Listening — 
Reading .84 
Structure .82 .92 



Table 5 indicates that there is a fairly strong relationship among the skills tested by the three 
subsections of the test. This relationship is strongest between reading and structure, which may 
be expected in a highly inflected language as Polish. Nevertheless, it is reasonable to say that 
each subsection scores provides some unique information about the examinee's proficiency in 

13 
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the Polish language. This pattern of intercorrelations between the test scores helps support the 
validity of the constructs the PPT claims to measure. 

For Further Information 

As further data becomes available, CAL intends to update the statistics presented in this 
manual and supply with additional information pertinent to users of the PPT. In the interim, any 
questions or comments about the PPT may be directed to: Coordinator, Polish Proficiency Test, 
Center for Applied Linguistics, 1118 22nd Street, NW, Washington, DC, 20037. 
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Appendix A 
Standard Error of Estimation 
Fnlkh Profiomcy Tcrt 
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Standard Error of Estimation 
Polish Proficiency Test 



Scaled Score 


Listening 




Structure 
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Appendix B 

Comprehension Tasks for the PPT Items 

Each item in the listening and reading comprehension section was classified by the item 
developers as to the type of comprehension task required to answer the item correctly. Three 
broad areas of comprehension were used: understanding the main idea, understanding a fact or 
detail, or understanding an inference. 

Items identified as requiring the examinee to understand the main idea typically require the 
general comprehension of the entire passage. In same cases, the main gist of the passage may 
be grasped with the comprehension of one or two key words or phrases. In the majority of the 
items, however, comprehension of the main idea comes only if most of the passage is correctly 
understood. 

Items identified as requiring the examinee to understand a fact or supporting detail generally 
require comprehension of the main idea and some supporting detail. However, in some non- 
prose items from authentic materials (e.g., advertisements), an item in this category may entail 
the comprehension of isolated words. 



Items identified as requiring the examinee to understand an inference require a thorough 
comprehension of the entire passage in order to identify a correct implication based on the 
passage but which is not stated directly in it. 



Comprehension Task 


Listening Section 


Reading Section 


Understand the Main Idea 


23 items 


20 items 


Understand a Fact or 
Supporting Detail 


20 items 


26 items 


Understand an Inference 


7 items 


4 items 
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Topic Areas Covered on the PPT 



Each passage used in the listening and reading comprehension section was classified by the 
test developers in terms of the topic area which best described its content and vocabulary 
domain. Since the PPT is a test of general proficiency in Polish, these «opic areas are to be 
understood broadly. In other words, no examinee would need particular expertise in any of 
these topic areas to answer correctly an item related to that topic area. On the other hand, this 
table gives an indication of the distribution of the general vocabulary and content found on the 
PPT in terms of broad categories. Although there is often much overlapping between categories, 
each item has been placed in only 1 category in this table. 



| Topic Area 


Listening Section 


Reading Section | 


Art 


1 




Business 


1 


2 


Consumerism 


7 


6 


Dates 


1 





Economics/the Economy 


3 


5 


Education 


3 


6 


Entertainment 


5 


5 


Everyday Life 


3 


1 


Family 


3 


1 


Food 


2 
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Geography 


— 
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Government 


1 


1 


Health 
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History 


— 
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Housing 
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Information Listing 


— 
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Politics 


3 
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Social Exchange 


3 




Social Life 




2 


Society 


1 


1 


1 Sports 


4 


2 


| Transportation 


3 


7 


| Weather 


1 




1 Other 




3 
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Speaking Functions found in the PPT listening Comprehension Passages 



Each passage in the listening comprehension section was classified by the test developers in 
terms of the primary speaking function of the passage. In inany cases, there are several 
speaking functions. Each item, however, was assigned to only one. Generally, it is the one 
most connected with getting the item for the passage correct. Thus this table is meant to give 
an indication of the distribution of the speaking functions found in the PPT listening 
comprehension section. 



Speaking Function 


Number of Items 


Complaining 


6 


Describing 


1 


Expressing an opinion 


2 


Giving advice 


1 


Giving information 


19 


Making an inquiry 


11 


Interviewing 


1 


Making a comment 


1 


Making a request 




Making a purchase 




Making an introduction 




Making an invitation 




Negotiating 




Persuading 




Showing preferences 
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Grammatical Areas Covered in the PPT Structure Items 

Each item in the structure section of the PPT was classified as involving primarily either one 
part of speech or another aspect of structure (such as negation). This table shows the number 
of items involving those primary forms. Specific aspects of each form (such as case, number 
agreement, gender, tense) are not listed separately. 



Form 


Number of Items 


Nouns 


11 


Verbs 


7 


Adjective/ Adverbs 


6 


Pronouns 


5 


Prepositions 


3 


Other 


3 
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Sources for PPT Passages 

This table lists the most widely used types 

Listening EaSSflggS 

Advertisements 
Commentaries 
Interviews 
Movies 

Radio News Reports 
Radio Announcements 
Recorded Conversations 
Television News Reports 
Television Announcements 
Television Programming 
Weather Forecasts 



of sources for passages found on the PPT. 

Reading Passages 

Advertisements 

Bulletins 

Editorials 

Instructions 

Labels 

Letters 

Magazine Articles 
Newspaper Articles 
Popular Novels 
Schedules 
Signs 
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POLISH PROFICIENCY TEST 
Order Form 



mc IK BEST C8PY AVAILABLE 



Ship to: (Please print) Bill to: (if different from left) 



Name Name 



PositionVTitle Position/Title 



Department Department 



Institution Institution 



Street Street 



City/State/Zip Code City/State/Zip Code 



Telephone Number Telephone Number 

First order? 

Yea (If yes, CAL reserves the right to verify the above information.) 

No 

Nisnbcr of examinees expected: ($20 per examinee) 

Purpose and use of the test: (Check all that apply) 

admission to an Polish study program 

placement within an Polish study program 

exemption from an Polish language requirement 

application for scholarship or appointment 

competency testing upon exit from an Polish program 

certification of Polish language proficiency for career placement 

evaluation of an Polish instructional program 

other (Please specify: 



Approximate testing date: Required detiTery date: 

Payment: 

You will be billed for the actual number of examinees tested after you return the test materials to CAL. 
Delivery charges will be paid by CAL, provided this Order Form reaches CAL at least twn w^g prior to the 
required delivery date indicated above. Delivery charges for rush orders requiring a shorter delivery time will be 
billed to the institution placing the order. Official score reports will be issued about two weeks after the materi als 
have been returned to CAL. 

Security Agreement 

It is vital that you maintain the security of the test materials from the time you receive them until they are 
returned to CAL. You must agree to: (1) keep the PPT test booklets) and test tape(s) confidential and secure, and 
(2) guarantee that none of the test booklets) or test tape(s) will be reproduced in any form. Please signify your 
acceptance of these terms by signing your full name below. 



Signature Date 



Office Use Only 

Date received: Date filled: Processed bv: 
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POLISH PROFICIENCY TEST 
Order Form 



Ship to: (Please print) BUI to: (if different from left) 



Name Name 



Position Title Position Title 



Department Department 



Institution Institution 



Street Street 



City/State/Zip Code City/State/Zip Code 



Telephone # Telephone # 



First order 



Yes (If yes, CAL reserves the right to verify the above information) 
No 



Number of examinees expected: ($20 per examinee) 

Purpose and use of the test: (Check all that apply) 

admission to a Polish study program 

placement within a Polish study program 

exemption from Polish language requirement 

application for scholarship or appointment 

competency testing upon exit from a Polish program 

certification of Polish language proficiency for career purposes 

other (Please specify: 



Approximate testing date: Required delivery date: 

Payment 

You will be billed for the actual number of students tested after you return the test materials to CAL. Deliver} 
chargev will be paid b> CAL. provided this order form reaches CAL at least two weeks pnor to the required delivery 
date indicated above Delivery charges for rush orders requiring a shorter delivery time will be billed to the 
institution placing the order. Official score reports will be issued about two weeks after the materials have been 
returned to CAL 

Securitv Agreement 

It is vital that >ou maintain the secunt> of the test materials from the time you receive tbem until the> are 
returned t,. CAL You must agree to: (1) keep the PPT test booklet(s) and test tape<s) confidential and secure! and 
(2) guarantee that none of the test bookJet(s) or test tapefs) will be reproduced in any form. Please signif) your 
acceptance of these terms by signing your full name below. 



Signature ;r- 
* Date 

Office Use Onl> 

Date received Date filled ' Processed b\ 
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POLISH PROFICIENCY TEST 
(Test Order Kit) 

Supervisor: Date; 

Institution: Test Center Code: 

RECEIVING TEST MATFRTAT *• 

The following materials should be contained in this package. Please check all materials received 
™ nSt .? JS Ust - V thcrc * *"> discrepancy, please contact the Polish Proficiency Test Office 
(202/429-9292) immediately. An* marks those items which must be returned. 

^ ; Enclosed 

Description Quantity in return 

- envelope? 
•1. Test Materials Checklist 

2. Order Form 

•3. Supervisor's Manual 

4. Examinee Handbook 

•5. Polish Proficiency Test Booklet 

(Serial numbers to ) 

•6. Polish Proficiency Test Tape 

(Serial numbers to ) 

•?. Standard Answer Sheet 

•8. Supervisor's Report Form 

9. Test Interpretation Manual 

10. Pre-addressed First -Class Return Envelope 

IMPORTANT: The Supervisor's Manual must be carefully reviewed prior to the administration 
date. The Examinee Handbook(s) should be distributed to students well ahead of the testing 

RETURNING TEST MATERIALS: 

Withm three days after the test has been administered, the materials marked with an asterisk 
above must be returned to the Center for Applied linguistics. For each item returned please 
mark a check on this form and enclose it in the return envelope. Make sure the quantity of each 
item matches what was sent in the original shipment. Thank you. 

Office Use Only '. 



Datf sent: Date returned : 
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POLISH PROFICIENCY TEST 
Supervisor's Report Form 



This report form is used for reporting any irregularities in test administration and for providing 
any comments or suggestions you may have for improving the Polish Proficiency Test. Please 
fill out this form, sign your name, and return it with the test materials. Thank you. 



Test Security 

By agreeing to serve as the test supervisor, I am responsible for ensuring the security of 
the test. I have kept the test materials confidential and secure at all times. None of the test 
booklets or test tapes have been reproduced in any form. 

Irregularities: 

Test Administration 

The test was administered in exact accordance with the procedures described in the 
Supervisor's Manual. Any deviations from the stated procedures are listed below. 

Irregularities: 

Conditions of Test Materials 

Before returning the test materials, I have checked the condition of the test booklets and 
test tapes. All materials are being returned in their original condition. All materials listed on 
the Test Materials Checklist have been enclosed. 

Irregularities: 



Comments 'Suggestions 

(In order to continue to meet your testing needs, it is ver> important that you 
communicate with us any comments or suggestions you may have regarding the Polish 
Proficienc> Test. Please use the back of this form or attach a separate sheet if necessary.) 



Name Institution 



Signature Date 
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Minutes of First Local Development Committee Meeting 
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MINUTES 



LOCAL TEST DEVELOPMENT COMMITTEE 
POLI8B PROFICIENCY TEST PROJECT 

OCTOBER 18 and 19, 1991 
CENTER TOR APPLIED LINGUISTICS 
WASHINGTON, DC 



Committee Members and CAL Staff Present: $ 

Dobrochna E. Dyrcz-Freeman . . . (Formerly) Princeton University 

Rafal M. Kwiatkowski (Formerly) Georgetown University 

Grazyna z. Privorotsky .... CAL, George Washington University 

Krystyna M. Sadlowska Foreign Service Institute 

Waldemar E. Walczynski £ CAL 

Xixiang Jiang $ CAL 

Dorry Mann Kenyon !!!!!■ CAL 

Charles W. Stansfield ! ! ! £ CAL 

Introduction and Background 

The members of the Committee introduced themselves. The 
Committee is comprised of Polish language experts, who are 
experienced instructors and researchers. This group is assisted 
by a CAL Supervisory group. Charles Stansfield serves as Project 
Director; Dorry Kenyon serves as Assistant Project Director, and 
Xixiang Jiang is Project Coordinator. 

stansfield, in his introductory remarks, noted the increased 
interest in Polish language study and explained the need for a 
national standardized proficiency test. It is hoped that this 
test, once developed, will serve as a catalyst for the 
improvement of Polish language instruction and learning. 
Stansfield also described CAL's Chinese Proficiency Test (CPT) , 
which has been used by colleges and high schools across the 
nation. This proficiency test, along with its second form called 
Pre-CPT, has been well received in the Chinese language teaching 
profession both for its validity and applicability. The CPT will 
serve as a model for the development of the Polish Proficiency 
Test. 1 

Project Schedule 

Kenyon presented the time frame for the test development project. 
He no^ed the importance for the draft test form to be ready in 
time for field testing in January/February of 1992, and the final 
version to be normed before school breaks for vacation (May, 
1992). The first draft test items are expected to be ready by 
the middle of December. From then on, there will be various peer 
reviews and revisions. The External Advisory Committee member 
will complete their review of the items in January 1992. Field 
testing is planned for February. Members of the External 
Advisory Committee will be called upon to assist in arranging for 
field testing at their own institutions, if possible. 
Statistical analysis and revisions after field testing will be 
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done in March and April. The nonning form of the test will be 
administered in between April and May. In the months to follow, 
the final form of the Polish Proficiency Test will be assembled 
and operationalized. The Project Final Report will be completed 
by the end of September 1992. 



Target Levels 

In the discussion of the target levels for the Polish language 
test under development, several issues were addressed. The first 
issue was which proficiency level to choose as the target level 
for the test. In view of the dif t iculty of the language itself 
and the limited nature of the exirting teaching materials, it was 
agreed that the appropriate target levels for this test should 
range from Novice-High through Advanced-Plus on the proficiency 
scale of the American Council for the Teaching of Foreign 
Languages (ACTFL) . This range is equivalent to 0+ through 2 + on 
the proficiency scale of the Federal Interagency Language 
Roundtable (FILR) . The second issue was which student body to 
choose as the target group for this test. Noting that most 
Polish language teaching programs do not continue beyond the 
second year and that students in area studies at the graduate 
level usually enter beginning Polish programs av. the 
undergraduate level, it was agreed that the envisioned target 
student group for this language test includes Polish language 
students in their first or second year of college and government 
or business professionals the intensive language training 
programs . 



Test Domains 

The difference between an achievement test and a proficiency test 
was reviewed. It was recognized that there are several Polish 
language textbooks in wide use and that, due to the nature of the 
language itself, there is a strong emphasis on grammatical 
structures in the teaching profession, it was also recoqnized 
that since the test under development is to serve as a national 
standardized proficiency test, it should not be based on a 
particular set of textbooks or language programs, though efforts 
should be made to examine some of the books and programs during 
the test development process. Acknowledging that the knowledge 
of syntactic structures is an integral part of communicative 
ability in the Polish language, it was determined that this test 
will include a structure section, probably 20-25% (30 to 40 
items) of the entire test. There will be three sections in the 
finished test: Listening Comprehension, Reading Comprehension and 
Structure. 



Test Fo rmat and Item Numbers 

In conformity with the current trend in large scale standardized 
tests, the Polish language test will employ the multiple choice 
rather than open-ended answer format. Several item types were 
reviewed, including error recognition, sentence completion and 
lexical insertion. It was generally agreed that a four-option 
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choice test type be used throughout the test. In both the 
Listening and Reading Comprehension Sections, four answer 
choices will be provided for each test question. Two types of 
items were discussed as possibilities for the Structure Section. 
The first is completion, in which the examinee would need to 
choose the correct words that complete missing portions within a 
Polish text. The second item format was error recognition, in 
which the examinee must find the error in a line of Polish text 
(within a larger passage) . Four words in a line would be 
underlined, one of which contains a grammatical error. For the 
whole test, there will be between 125 and 150 items, with roughly 
equal numbers of Listning and Reading Comprehension items. 

Having laid down the above guidelines and specifications, the 
Committee proceeded to deliberate on issues pertaining to other 
aspects of the project. The following concerns were addressed. 

External Advisory Committee 

Considering the small number of people currently listed in the 
External Advisory Committee, members of the Local Test 
Development Committee felt that there is a need to increase the 
number and include Polish language professionals from a wider 
range of institutions. Names were suggested and noted. An 
External Advisory Committee with representatives from various 
educational and research backgrounds will enhance the test 
development efforts and render more professional assistance in 
the process of reviewing test items and locating Polish language 
programs across the country. 

Sources of Polish language Students 

In order to validate and norm the test after initial development, 
there needs to be enough students to participate in the field 
test and norming administration. The minimum preliminary number 
of students for the field test is 200. Members of the committee 
offered names of people who could provide suggestions, locations 
of various existing Polish language programs and institutions 
with potential sources of students. After recruiting efforts are 
undertaken, a mailing list will be established, not only for use 
in the test development process, but also for disseminating 
information dissemination on the Polish Proficiency Test once it 
is in its operational form. 

Sources of Authentic Materials 

It was quickly realized that materials for the Reading 
Comprehension are relatively easy to collect, since some members 
of the committee have been accumulating Polish newspapers and 
other authentic materials for other projects. It was noted that 
in addition to newspapers, item writers will collect other 
authentic materials in order to include language forms. These 
materials will be drawn from novels, stories, magazines, personal 
and public documents. Efforts will be made to collect realia, 
including objects or signs often encountered in a real life 
Polish situation. Gathering materials for the Listening 
Comprehension Section appeared to be not as easy, though some 
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sources were quickly identified. Government agencies, the 
embassy, radio and TV companies were mentioned for their 
collection of original audio tapes, videos of movies and tapes of 
broadcast programs. Items will be written using these authentic 
materials* However, the materials will be re-recorded by Polish 
professionals to avoid possible background noise and distraction. 
As was noted, for both reading and listening items, there will be 
some efforts required to collect authentic materials at the low 
proficiency levels specified earlier. 

Item Writing Procedures 

Once developed and approved, the test development guidelines and 
specifications will serve as the principal framework from which 
individual item writers will work. It was agreed that the item 
writers will mainly work at their own rate to collect source 
materials and write the assigned items. To ensure adequate 
communication and peer review, however, it was proposed that item 
writers submit their test items in keeping with the work schedule 
for review and approval. CAL staff will confer with the item 
writers on a regular basis in critiquing and evaluating the 
submitted items. At the development stage, work will be divided 
among the item writers. Each individual will be responsible for 
collecting authentic language materials. After the 
appropriateness of the selected materials has been determined by 
a process of peer review, each person will write the assigned 
test items. For the field test version, the number of test items 
will be approximately 180 in all. Item writers are expected to 
keep a record of the sources from which their items are composed. 

After deliberations on these issues, the Committee started an 
item writing training workshop. The item writers, with the 
assistance of Stansfield and Kenyon, reviewed some principles of 
test item writing, evaluated some sample test items written in a 
Polish textbook, and finally practiced writing some reading 
comprehension items. The peer review and critiquing process that 
followed proved to be informative and beneficial for the item 
writers. 

The meeting adjourned at 3:00 p.m. Saturday, October 19, 1991. 
No following meeting was scheduled, but members of the Committee 
expressed their availability for such an occasion, if needed. 

Respectfully submitted, 



Xixiang Jiang, Project Coordinator 



Dorry Mann Kenyon, Assistant Project Director 



Charles W. Stansfield, Project Director 
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KEADIMO COMPREHENSION ITEM SUBMISSION FORM 

Source 



Number: 



Text Form " 
Topic Area' 
Context 



Cognitive Task i To Understand 

( ) Learned Material ( ) Main Idea 

( ) Implication ( ) other: 



( ) Pacts/Detail 



Contextual Support 

Vocabulary 

Creamer 



( ) Strong 
( ) Easy 
( ) £*«y 



( ) Moderate 
( ) Medium 
( ) Medium 



( ) Weak 

( ) Difficult 
( ) Difficult 



Kay to Underetanding 

Level of Difficulty 
Stimulus (Polish) : 



( ) vocabulary 
( ) Context 
( ) Other: 



( ) Cramnar 
( > Pragmatics 



WL MM HH IL 



IM IH 



A* 



(Translation) 



Question or Stea: 
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Options: (* ■ Key) 
(A) 



(B) 
(C) 
(D) 

Author: Date: 
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LISTENING COMPREHENSION ITEM SUBMISSION FORM 
Source 



Number: 



Speaking Function(s) 

Topic Area 

Context 



Exchange Pattarn ( ) U ( ) A-B ( ) A-B-A ( ) A-B-A-B ( ) M 



Cognitive Task To Understand 



( ) Learned Material 
( ) Implication 

Contextual Support 
Vocabulary 
Grammar 



Key to Underatanding 

Level of Difficulty 
Stimulus (Polish) : 



( ) Main Idea 
( ) Other: 



( ) Pacts /Detail 



( ) Strong 
( ) Easy 
( ) Easy 



( ) Moderate 
( ) Medium 
( ) Medium 



( ) Weak 

( ) Difficult 
( ) Difficult 



( ) Vocabulary 
( ) Context 
( ) Other: 



( ) Grammar 
( ) Pragmatics 



NL 



NM NH IL IM IH 



A+ 



(Translation) 



Question: 

Options: (* «= Key) 
(A) 

(B) 

(C) 

(D) 



er|c 



Author: 
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PPT FIELD TESTING 
PPT BACKGROUND INFORMATION SHEET 



JL- In what kind of school do you presently receive Polish language instruction? 

1 - Public High School 

2 - Private High School 

3 - College or University 

4 — Community School/Weekend Language Program 

5 - Other 

l~ How many years of Polish language study. NOT INCLUDING THE CURRENT YEAR, 
have you completed? 

0 - None 3 - 3 years 

1-1 year 4 ■ 4 or more years 

2 - 2 years 

M: How many college course credits. NOT INCLUDING YOUR CURRENT COURSE, 
have you already earned in Polish? 

0 - None 

1 - 1 to 3 5 - 13 to 15 

2 '4 to 6 6' 16 to 18 

3 - 7 to 9 7 - 19 or more 

4 - 10 to 12 8 - NOT APPLICABLE 

iV: For how many hours a week does your current Polish language doss meet? 

0 - Not Applicable 5 - 5 hours/week 

1 - 1 hour/week 6 - 6 hours/week 

2 - 2 hours/week 7 - 7 hours/week 

3- 3 hours/week 8-8 hours/week 

4- 4 hours/week 9 - 9 or more hours/week 

O: Are you of Polish ethnic heritage? 

0- Yes 
1 - No 

P: Do you speak Polish at home? 

0 - Yes, most of the time 

1 - Yes, but less than half the time 

2 - Yes, but very infrequently 
3 -No 
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Standard Error of Estimation 
Polish Proficiency Test 



j Scaled Score I Listening 


Readme ] Structure 


171 


25 


26 


27 


170 


24 


25 


27 


169 


23 


25 


26 


168 


22 


24 


25 


167 


22 


23 


24 




21 


22 


23 


165 


20 


22 


23 


164 


20 


21 


22 


163 


19 


20 


21 


162 


18 


20 


21 


161 


18 


19 


20 


160 


17 


19 


19 


159 


17 


18 


19 


158 


16 


18 


18 


157 


16 


17 


18 


156 


15 


16 


I 7 


155 


15 


16 


17 


154 


14 


16 


16 


I 153 


14 


15 


16 




13 


15 


15 


8 151 


13 


14 


15 


150 


12 


14 


14 


149 


12 


13 


14 


148 


12 


13 


13 


14^ 


1 1 


13 


13 


146 


1 1 


12 


13 


145 


11 


12 


12 


144 


10 


12 


12 


143 


10 


11 


12 


142 


10 


11 


1 1 


14] 


10 


1 1 


1 1 


140 


9 


10 


1 1 


139 


9 


10 


10 


138 


9 


10 


10 


13" 


9 


10 


10 


136 


8 


9 


9 


135 


8 


9 


9 


134 


8 


9 


9 


m 


8 


9 


9 


132 I 8 


8 


9 


131 


7 


8 


9 


130 


•7 


8 


8 


129 




8 


8 


128 


1 


8 


8 


12 n 


-» 


8 


8 


126 




7 


8 


125 




7 


7 


124 


6 


n 


7 


123 


6 


1 


7 


122 


6 


1 




121 


6 




*7 


12(» 


6 


n 




1 19 


6 


6 




1 IK 


6 


6 




i r 


6 


6 


6 
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Standard Error of Estimation 

Polish Proficiency Test 





l^isicnini: 




Structure 


116 


6 


6 


6 


115 


6 


6 


6 


114 


5 


6 


6 


113 


5 


6 


6 


112 


5 


6 


6 


111 


5 


6 


6 


110 


5 


6 




109 


5 


5 


6 

V 


108 


5 


5 


5 


107 


5 


5 


A 

V 


106 


5 


5 




105 


5 


5 


6 


104 


5 


5 


6 


103 


5 


5 


5 


102 


5 


5 


5 


101 


5 


5 


5 


100 


5 


5 


5 


99 


5 


5 


5 


98 


5 


5 


5 


97 


5 


5 


5 


96 


5 


5 


5 


95 


5 


5 


5 


94 


5 


5 


5 


93 


5 


5 




92 


5 


5 


5 


91 


5 


5 




90 


5 


5 


5 


89 


5 


5 


6 


88 


5 


5 


6 


8* 1 


5 


5 


6 


86 


5 


5 


6 


85 


5 


5 


6 


84 


5 


5 


6 


83 


5 


5 


6 


82 


5 


5 


6 


81 


5 


5 




80 


5 


5 


u 


79 


5 


5 


D 


78 


5 


5 




77 


5 


5 




76 


5 


5 


6 


75 


5 


5 




74 


5 


5 




^3 1| 5 


5 




7** 


5 


5 




71 


5 


5 




70 


5 


5 




6<v 


5 


5 


7 


6fc 


5 


5 




6" 1 


5 


6 


6 


66 


5 


6 
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Comprehension Tasks for the PPT Items 



Each item in the listening and reading comprehension section was classified by the item 
developers as to the type of comprehension task required to answer the item correctly. Three 
broad areas of comprehension were used: understanding the main idea, understanding a fact 
or detail, or understanding an inference. 

Items identified as requiring the examinee to understand the main idea typically require 
the general comprehension of the entire passage. In same cases, the main gist of the passage 
ma> be grasped with the comprehension of one or two key words or phrases. In the majont\ 
of the items, however, comprehension of the main idea comes only if most of the passage is 
conectlv understood. 

Items identified as requiring the examinee to understand a fact or supporting detail 
generally require comprehension of the main idea and some supporting detail. However, in 
some non-prose items from authentic materials (e.g., advertisements), an item in this 
category ma\ entail the comprehension of isolated words. 



Items identified as requiring the examinee to understand an inference require a thorough 
comprehension of the entire passage in order to identify a correct implication based on the 
passage but which is not stated directly in it. 



Comprehension Task 


Listening Section 


Reading Section 


Understand the Main Idea 


23 items 


20 items 


1 Understand a Fact or 
j Supporting Detail 


20 items 


26 items 


[ Understand an Inference 


7 items 


4 items 
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Grammatical Areas Covered in the PPT Structure Items 



134 



Grammatical Areas Covered in the PPT Structure Items 

Each item in the structure section of the PPT was classified as involving primarily either 
one part of speech or another aspect of structure (such as negation). This table shows the 
number of uems involving those primary forms. Specific aspects of each form (such as case, 
number agreement, gender, tense) are not listed separately. 



Form 


Number of Items 


Nouns 


11 


Verbs 


7 


Adjective/ Adverbs 


6 


Pronouns 


5 


Prepositions 


3 


Other 


3 
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Sources for PPT Passages 



This table lists the most widely used 
Listening Passages 

Advertisements 
Commentaries 
Interviews 
Movies 

Radio News Reports 
Radio Announcements 
Recorded Conversations 
vision News Reports 
K ^vision Announcements 
television Programming 
Weather Forecasts 



of sources for passages found on the PPT. 
Reading Passages 

Advertisements 

Bulletins 

Editorials 

Instructions 

Labels 

Letters 

Magazine Articles 
Newspaper Articles 
Popular Novels 
Schedules 
Signs 



ERIC 



137 



Appendix N 
PPT Announcement 



138 

ERJC 




Center fcx 

Applied 

Linguistics 



CAL Announces the Development of the Polish Proficiency Tost 



With funding from the U.S. Department of Education, the Center 
for Applied Linguistics (CAL) is currently developing a national 
standardized test in the Polish language. This test, vhich will 
include listening and reading comprehension sections, will be 
known as the Polish Proficiency Test (PPT)* When in operation, 
this test is expected to help strengthen Polish language 
instruction and learning in the United States by serving as a 
tool to both assess student progress across programs and evaluate 
the effectiveness of individual programs. 

The PPT Development Project, initiated in October, 1991, consists 
of three working groups: an External Advisory Committee, a Local 
Test Development Committee and a Project Supervisory Group. 
These three groups include a wide variety of individuals with 
considerable background and stature in the field of Polish 
language instruction, foreign language education and testing. 
Dr. Charles W. Stansfield, Director of Division of Foreign 
Language Education and Testing at CAL, serves as the Project 
Director. 

The Polish Proficiency Test is expected to be available by the 
end of September, 1992. As of December, 1991, it is in the 
process of item development. In February, 1992, the field test 
version of the PPT will be administered at different sites 
nationwide. In April and May, 1992, the final version will be 
administered for norming purposes. 

Since the PPT is designed to be a general proficiency test and 
not an achievement test, and since it is being developed with 
authentic materials, it is hoped that teachers and students in 
Polish language teaching programs will benefit from participating 
in the developmental testing and by using the PPT once it is in 
operation. If you need more information about the Polish 
Proficiency Test, or if you would like to have your program 
participate in its testing in February, or in April and May, 
please contact: 



Please note that all development testing is done free of charge 

to the participating institutions. 



Mr. Xixiang Jiang 



PPT Project Coordinator 



Center For Applied Linguistics 



1118 22nd Street, N.W. 
Washington D.C. 20037 
Telephone: (202)429-9292 



1118 22nd Street NW Washington DC 20G3 7 
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(202' 429-9292 



Fat (202 1 659-5641 
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